Keyword Analytics

TO DOs

- Generell sind Fragen an dich in den Chunks (glaub auch nur beim Data Load) mit ??? gekennzeichnet, erleichter die Suche ;)

- Sixtrix hat nur sehr wenige Datenpunkte/Keywords (2,368/10,000) -> ist halt so, wird dann vermerkt.

* DONE Lollipops (VOL, DIF, CPC) per Volume Category

* DONE Remove DIF from Lollipops

* Heatmap with largest discrepancy

* DONE One Overview Plot: How many Observations per Category (KWS, VOL, DIF, CPC) and missing?

* How to deal with min ranked keywords? Try again for DIF + CPC! (also makes more sense than volume)

* DONE Dot Plot: Larger differences for small values (log? manual?)

* DONE Adjust row/col Facet Keywords by Length

* Length: Beispielrechnung (e.g. ein Character weniger führt zu x mehr Volume)

* DONE Move to Appendix: - DONE Histograms - DONE Tables - DONE Linear Jitter Plots - DONE Bar Plots - DONE Keywords by Length

-> 4-5 finale Grafiken
-> ich highlighte mal meine Favorites

## save plots?
save <- TRUE
#save <- FALSE

## packages
required_packages <- c("tidyverse", "readxl", "ggthemes", "hrbrthemes", "extrafont", "plotly", "scales", "stringr", "gganimate", "here", "tidytext", "sentimentr", "scales", "DT", "here", "sm", "mblm", "prettydoc", "reshape2", "treemapify", "glue", "magick", "imager", "fs", "knitr", "DataExplorer", "inspectdf", "rmdformats", "prettydoc", "janitor", "showtext", "ggbeeswarm", "ggtext", "kableExtra", "rcartocolor", "ggsci", "patchwork")

for(i in required_packages) { 
  if(!require(i, character.only = T)) {
    #  if package is not existing, install then load the package
    install.packages(i, dependencies = T)
    library(i, character.only = T)
  }
}

## theme updates
font_add_google("Montserrat", "Montserrat")

theme_set(ggthemes::theme_clean(base_size = 15, base_family = "Montserrat"))

theme_update(plot.background = element_rect(color = NA),
             #panel.background = element_rect(color = "grey20", size = .4), ## turn off if lighter version
             plot.title = element_markdown(size = 18, hjust = .5, face = "plain"),
             plot.subtitle = element_text(size = 12, hjust = .5, face = "plain"),
             axis.line.x = element_line(color = "grey20"),  ## turn off if full panel border
             axis.line.y = element_line(color = "grey20"),  ## turn off if full panel border
             #axis.line.x = element_blank(), axis.line.y = element_blank(), ## turn off if lighter version
             axis.ticks = element_line(color = "grey20", size = .3),
             axis.title.x = element_text(face = "plain", margin = margin(t = 10)),
             axis.title.y = element_text(face = "plain", margin = margin(r = 10)),
             axis.text = element_text(color = "grey20"),
             legend.title = element_text(color = "grey33"),
             legend.text = element_text(family = "Montserrat", size = 10, color = "grey33"),
             legend.background = element_rect(fill = NA, color = NA))

## theme settings for flipped plots
theme_flip <- 
  theme(panel.grid.major.x = element_line(colour = "gray", linetype = "dotted", size = .25),
        panel.grid.major.y = element_blank(),
        axis.title = element_text(face = "bold"),
        axis.text.y = element_text(face = "bold"),
        legend.position = "bottom")

## theme settings for facet plots
theme_facet <-
  theme(axis.text.y = element_text(face = "plain", size = 10),
        axis.text.x = element_text(size = 8),
        strip.text = element_text(face = "bold"),
        plot.margin = margin(10, 10, 10, 25),
        panel.spacing = unit(1, "lines"))

## theme settings for lollipop plots
theme_lolli <- 
  theme(axis.title.x = element_text(size = 13, color ="grey20"),
        axis.text.x = element_text(size = 11),
        axis.text.y = element_blank(),
        axis.line.y = element_blank(),
        axis.ticks.y = element_blank(),
        panel.grid.major.x = element_line(size = 0.6, color = "grey70"),
        plot.margin = margin(5, 20, 5, 20),
        plot.background = element_rect(fill = NA, color = NA))

## numeric format for labels
num_format <- format_format(big.mark = ",", small.mark = ",", scientific = F)

1 Introduction

There is a wealth of Software-as-a-Service (SaaS) companies today that offer a range of keyword analytics data, such as keyword search volume and keyword difficulty scores. For the end user, it is often challenging to make informed comparisons between different data providers, which may be skewed or misleading when viewed individually.

This large-scale analysis of Keyword Analytics data aims to compare data across various SEO tools.More specificially we will do a comparision of the following variables:

• Keyword Suggestions
• Average Monthly Search Volume
• Keyword Difficulty
• Cost-per-Click Ads data (in US dollar)

We compare the data of a total of 10 tools:

• Google Keyword Planner
• Ahrefs
• SEMrush
• Moz
• KeywordTool.io
• KWfinder
• LongtailPro
• SEOCockpit
• Sixtrix
• Ubersuggest (only Keyword Suggestions)
• Majestic (only Keyword Suggestions???)

METHODOLOGY PART -> Daniel

## path to processed data
rmd_tools <- here("proc_data", "tools.Rmd")

## if data is not available, load, clean and bind datasets
if(!file.exists(rmd_tools)){
  ### load data #################################################################
  
  ## Google Keyword Planner Data (Tool #1)
  df_gkp_raw <- read_csv(here("raw_data", "Step3_Tools_data", "1_Google_keyword_Planer_basis", "GKP_data_final_split.csv"))
  
  ## Ahrefs Data (Tool #2)
  df_ahr_raw <- read_csv(here("raw_data", "Step3_Tools_data", "2_Ahrefs", "export", "ahrefs_export.csv"))
  ## contains less rows than GKP data as the tool does not have enough data points. Should go into the analysis.
  
  ## Moz (Tool #3)
  path <- here("raw_data", "Step3_Tools_data", "3_Moz", "export")
  file_names_csv <- list.files(path = path, recursive = T, full.names = T) 
  file <- map(file_names_csv, 
              function(x) read_csv(x, skip = 13) %>% 
                            mutate(`Min Volume` = as.numeric(`Min Volume`),
                                   `Max Volume` = as.numeric(`Max Volume`))) 
  names(file) <- gsub(".csv","", list.files(path, full.names = F), fixed = T)
  
  df_moz_raw <- 
    bind_rows(file, .id = "column_name") %>% 
    clean_names() 
  
  ## SEMRush (Tool #4)
  path <- here("raw_data", "Step3_Tools_data", "4_SemRush", "export")
  file_names_csv <- list.files(path = path, recursive = T, full.names = T) 
  file <- map(file_names_csv, read_csv) 
  names(file) <- gsub(".csv","", list.files(path, full.names = F), fixed = T)
  
  df_sem_raw <- 
    bind_rows(file, .id = "column_name") %>% 
    clean_names() 
  
  ## KeywordTool.io (Tool #5)
  path <- here("raw_data", "Step3_Tools_data", "5_Keywordtool", "export")
  file_names_csv <- list.files(path = path, recursive = T, full.names = T) 
  file <- map(file_names_csv, read_csv) 
  names(file) <- gsub(".csv","", list.files(path, full.names = F), fixed = T)
  
  df_kwst_raw <- 
    bind_rows(file, .id = "column_name") %>% 
    clean_names() 
  
  ## KWFinder (Tool #6)
  path <- here("raw_data", "Step3_Tools_data", "6_KWfinder", "export")
  file_names_csv <- list.files(path = path, recursive = T, full.names = T) 
  file <- map(file_names_csv, read_csv) 
  names(file) <- gsub(".csv","", list.files(path, full.names = F), fixed = T)
  
  df_kwsf_raw <- 
    bind_rows(file, .id = "column_name") %>% 
    clean_names() 
  
  ## LongtailPro (Tool #7)
  path <- here("raw_data", "Step3_Tools_data", "7_LongtailPro", "export")
  file_names_csv <- list.files(path = path, recursive = T, full.names = T) 
  file <- map(file_names_csv, read_csv) 
  names(file) <- gsub(".csv","", list.files(path, full.names = F), fixed = T)
  
  df_ltp_raw <- 
    bind_rows(file, .id = "column_name") %>% 
    clean_names() 
  
  ## SECockpit (Tool #8)
  path <- here("raw_data", "Step3_Tools_data", "8_SECockpit", "export")
  file_names_csv <- list.files(path = path, recursive = T, full.names = T) 
  file <- map(file_names_csv, function(x) read_csv2(x)) 
  names(file) <- gsub(".csv","", list.files(path, full.names = F), fixed = T)
  
  df_sec_raw <- 
    bind_rows(file, .id = "column_name") %>% 
    clean_names() 
  
  ## Sixtrix (Tool #9)
  path <- here("raw_data", "Step3_Tools_data", "9_Sixtrix", "export")
  file_names_csv <- list.files(path = path, recursive = T, full.names = T) 
  file <- map(file_names_csv, read_csv2) 
  names(file) <- gsub(".csv","", list.files(path, full.names = F), fixed = T)
  
  df_six_raw <- 
    bind_rows(file, .id = "column_name") %>% 
    clean_names() 
  
  
  ### clean data ################################################################
  
  ## select relevant columns and bind all tools
  ## Google Key Planner
  df_gkp <-
    df_gkp_raw %>% 
    mutate(
      tool_id = 1,
      tool = "Google Keyword Planner"
    ) %>% 
    dplyr::select(
      tool_id, tool,
      keyword,
      volume = "avg_monthly_searches.x",
      cpc_min = "top_of_page_bid_low_range.y",
      cpc_max = "top_of_page_bid_high_range.y"
    ) %>% 
    mutate(
      diff = NA_real_,  ## GKP dies not have a difficulty score output
      cpc = cpc_max - (cpc_max - cpc_min) / 2     
    ) %>% 
    dplyr::select(-cpc_min, -cpc_max)
  
  ## Ahrefs
  df_ahr <-
    df_ahr_raw %>% 
    mutate(
      tool_id = 2,
      tool = "Ahrefs"
    ) %>% 
    dplyr::select(
      tool_id, tool,
      keyword = "Keyword",
      volume = "Volume",
      diff = "Difficulty", 
      cpc = "CPC"
    ) 
  
  ## Moz
  df_moz <-
    df_moz_raw %>% 
    mutate(
      tool_id = 3,
      tool = "Moz (mix of sources)"
    ) %>% 
    dplyr::select(
      tool_id, tool,
      keyword,
      max_volume, 
      min_volume,
      diff = "difficulty"
    ) %>% 
    mutate(
      cpc = NA_real_,  ## Moz has no CPC output
      volume = round(max_volume - (max_volume - min_volume) / 2, 0)
    ) %>% 
    dplyr::select(-max_volume, -min_volume)
  
  ## SEMrush
  df_sem <-
    df_sem_raw %>% 
    mutate(
      tool_id = 4,
      tool = "SEMrush"
    ) %>% 
    dplyr::select(
      tool_id, tool,
      keyword,
      volume,
      diff = "keyword_difficulty", 
      cpc = "cpc_usd"
    ) 
  
  ## KeywordTool.io
  df_kwst <-
    df_kwst_raw %>% 
    mutate(
      tool_id = 5,
      tool = "KeywordTool.io"
    ) %>% 
    dplyr::select(
      tool_id, tool,
      keyword = "keywords",
      volume = "search_volume_average",
      diff = "competition", 
      cpc = "cpc_usd"
    ) 
  
  ## KWFinder
  df_kwsf <-
    df_kwsf_raw %>% 
    mutate(
      tool_id = 6,
      tool = "KWFinder"
    ) %>% 
    dplyr::select(
      tool_id, tool,
      keyword,
      volume = "avg_search_volume",
      diff = "keyword_difficulty", 
      cpc
    ) 
  
  df_ltp <-
    df_ltp_raw %>% 
    mutate(
      tool_id = 7,
      tool = "LongtailPro"
    ) %>% 
    dplyr::select(
      tool_id, tool,
      keyword = "keywords",
      volume,
      diff = "avg_kc", 
      cpc = "bid"
    ) %>% 
    mutate(cpc = as.numeric(str_sub(cpc, 2)))
  
  ## SECockpit
  df_sec <-
    df_sec_raw %>% 
    mutate(
      tool_id = 8,
      tool = "SECockpit"
    ) %>% 
    dplyr::select(
      tool_id, tool,
      keyword = "phrase",
      volume = "monthly_searches",
      diff = "top_results", 
      cpc
    ) %>% 
    mutate(
      cpc = as.numeric(cpc),
      diff = diff / max(diff) * 100  ## ??? top_results has a range from 6 to 464. Not sure hwat the possible max is so I take 464 for now
    )  
  
  ## Sixtrix
  df_six <-
    df_six_raw %>% 
    mutate(
      tool_id = 9,
      tool = "Sixtrix"
    ) %>% 
    dplyr::select(
      tool_id, tool,
      keyword = "x_u_feff_keyword",
      volume = "search_volume",
      diff = "competition",
      cpc
    ) %>% 
    mutate(
      ## extract max volume from range string
      volume = str_replace(volume, ".*\\s", ""),  ## using the max of the range here
      volume = str_replace(volume, "\\.", ""),
      thousands = if_else(str_detect(volume, ".*K"), T, F),  ## dealing with "K" units
      volume = str_replace(volume, "K", ""),
      volume = if_else(thousands == T, 
                       as.numeric(volume) * 1000, 
                       as.numeric(volume)),
      ## turn difficulty into numbers
      diff = as.numeric(diff), ## turns "-" to NA - guess that's good!
      ## clean cpc and convert to USD
      cpc = str_replace(cpc, ",", "."),
      cpc = str_sub(cpc, 1, -4),
      cpc = as.numeric(cpc) * 1.11 ## EUR -> USD, exchange rate 2020-01-16 05:46 pm
    ) %>% 
    dplyr::select(-thousands)
  
  
  ### final dataset #############################################################
  
  ## join all tools
  df_tools <-
    df_gkp %>% 
    bind_rows(df_ahr) %>% 
    bind_rows(df_moz) %>% 
    bind_rows(df_sem) %>% 
    bind_rows(df_kwst) %>% 
    bind_rows(df_kwsf) %>% 
    bind_rows(df_ltp) %>% 
    bind_rows(df_sec) %>% 
    bind_rows(df_six) %>% 
    mutate(
      tool = factor(tool, levels = unique(tool)),
      volume_cat = case_when(
        volume <= 100 ~ "0-100",
        volume > 100 & volume <= 1000 ~ "100-1000",
        volume > 1000 & volume <= 10000 ~ "1000-10000",
        volume > 10000 ~ "10000+",
      )
    ) %>% 
    filter(cpc > 0) ## remove $0 "costs"
  
  saveRDS(df_tools, file = rmd_tools)

}else{
  df_tools <- readRDS(rmd_tools)
}

## long format with volume, difficulty + cpc as one column 'cateogry'
df_tools_long <-
  df_tools %>% 
  dplyr::select(-tool_id) %>% 
  pivot_longer(c(-tool, -keyword, -volume_cat),
               names_to = "category",
               values_to = "value")

## path to processed data
rmd_kws <- here("proc_data", "keyword_suggestions.Rmd")

## load version 1 csv
df_kws_raw <- read_csv(here("raw_data", "Step3_Tools_data", "keyword_suggestions", "keyword_suggestions_v1.csv")) %>% 
  clean_names() 

if(!file.exists(rmd_kws)){
  
  ### clean data ################################################################
  
  ## Transform data into long format (two columns with *SEO tool* (`tool`) 
  ## and *suggested keywords* (`suggested`))
  df_kws <-
    df_kws_raw %>% 
    mutate_at(vars(se_mrush_broad_match:majestic), as.numeric) %>% 
    pivot_longer(
      cols = google_keyword_planner:majestic,
      names_to = "tool",
      values_to = "suggested"
    ) %>% 
    ## remove not avilable values
    filter(!is.na(suggested)) %>%
    ## remove combination of keyword search for now (since not all provide it)
    filter(!str_detect(keyword, "All 5 KW")) %>% 
    ## Clean SEO tool names
    mutate(tool = 
      factor(
        tool,
        levels = c("google_keyword_planner", 
                   "ahrefs_all_keyword_ideas", 
                   "ahrefs_phrase_match", 
                   "se_mrush_broad_match", 
                   "se_mrush_phrase_match", 
                   "moz_include_a_mix_of_sources", 
                   "ubersuggest_related", 
                   "se_cockpit", 
                   "keyword_tool_io", 
                   "kw_finder"),
        labels = c("Google Keyword Planner", 
                   "Ahrefs (all keyword ideas)", 
                   "Ahrefs (phrase match)", 
                   "SEMrush (broad match)", 
                   "SEMrush (phrase match)", 
                   "Moz (mix of sources)", 
                   "Ubersuggest (related)", 
                   "SECockpit", 
                   "KeywordTool.io", 
                   "KWFinder")
        )
    ) %>% 
    ## Sort SEO tools and keyword categories by mean number of suggestions
    group_by(tool) %>% 
    mutate(tool_median = median(suggested)) %>% 
    group_by(category) %>% 
    mutate(category_median = median(suggested)) %>% 
    ungroup() %>% 
    mutate(
      tool_fct = fct_reorder(tool, tool_median),
      category_fct = fct_reorder(category, category_median),
    )
  
  ### final dataset ########################################################
  
  
  saveRDS(df_kws, file = rmd_kws)
  
}else{
  df_kws <- readRDS(rmd_kws)
}

## range of suggestions
range_kw <- range(df_kws$suggested)

## number of keywords per cateory 
## (based on the un-cleanded data since we have removed NAs)
n_kw_cat <-
  df_kws_raw %>% 
  group_by(category) %>% 
  count() %>% 
  ungroup() %>% 
  summarize(n = unique(n)) %>% 
  pull(n)

dat_kws <-
  df_kws %>% 
  mutate(count_na = sum(is.na(suggested))) %>% 
  group_by(tool, count_na) %>% 
  summarize(n = n()) %>% 
  ungroup() %>% 
  mutate(tool_fct = fct_reorder(tool, n)) %>% 
  ggplot(aes(tool_fct, n)) +
    geom_col(aes(x = tool_fct, 75, color = tool), fill = "grey90", size = 1.1, width = 0.7) +
    geom_col(aes(fill = tool), color = NA, width = 0.7, alpha = 0.6) +
    geom_text(aes(tool_fct, n - 1, label = comma(n, accuracy = 1)), 
              color = "white", fontface = "bold", family = "Montserrat", size = 2.5, hjust = 1) +
    coord_flip() +
    scale_y_continuous(expand = c(.005, .005),
                       breaks = seq(0, 75, by = 25)) +
    scale_color_carto_d(palette = "Prism", guide = F) +
    scale_fill_carto_d(palette = "Prism", guide = F) +
    labs(x = NULL, y = "", title = "**Successful Search Requests per SEO Tool**<br>", subtitle = "Keyword Suggestions") +
    theme_flip 
  

dat_vol <-
  df_tools_long %>% 
  filter(category == "volume") %>% 
  group_by(tool) %>% 
  summarize(n = sum(!is.na(value))) %>% 
  ungroup() %>% 
  mutate(tool_fct = fct_reorder(tool, n)) %>% 
  ggplot(aes(tool_fct, n)) +
    geom_col(aes(tool_fct, 10000, color = tool), fill = "grey90", size = 1.1, width = 0.7) +
    geom_col(aes(fill = tool), color = NA, width = 0.7, alpha = 0.6) +
    geom_text(aes(tool_fct, n - 90, label = comma(n, accuracy = 1)), 
              color = "white", fontface = "bold", family = "Montserrat", size = 2.5, hjust = 1) +
    coord_flip() +
    scale_y_continuous(expand = c(.005, .005),
                       labels = num_format,
                       breaks = seq(0, 10000, by = 2000)) +
    scale_color_carto_d(palette = "Bold", guide = F, limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, limits = levels(df_tools$tool)) +
    labs(x = NULL, y = "", subtitle = "Keyword Search Volume") +
    theme_flip 
  
dat_dif <-
  df_tools_long %>% 
  filter(category == "diff") %>% 
  group_by(tool) %>% 
  summarize(n = sum(!is.na(value))) %>% 
  ungroup() %>% 
  mutate(tool_fct = fct_reorder(tool, n)) %>% 
  ggplot(aes(tool_fct, n)) +
    geom_col(aes(tool_fct, 10000, color = tool), fill = "grey90", size = 1.1, width = 0.7) +
    geom_col(aes(fill = tool), color = NA, width = 0.7, alpha = 0.6) +
    geom_text(aes(tool_fct, n - 90, label = comma(n, accuracy = 1), alpha = (n > 0)), 
              color = "white", fontface = "bold", family = "Montserrat", size = 2.5, hjust = 1) +
    geom_text(aes(tool_fct, n + 90, label = "not available", color = tool, alpha = n == 0), 
              fontface = "bold", family = "Montserrat", size = 2.5, hjust = 0) +
    coord_flip() +
    scale_y_continuous(expand = c(.005, .005),
                       labels = num_format,
                       breaks = seq(0, 10000, by = 2000)) +
    scale_color_carto_d(palette = "Bold", guide = F, limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, limits = levels(df_tools$tool)) +
    scale_alpha_manual(values = c(0, 1), guide = F) +
    labs(x = NULL, y = "", subtitle = "Difficulty Scores") +
    theme_flip 
  
dat_cpc <-
  df_tools_long %>% 
  filter(category == "volume") %>% 
  group_by(tool) %>% 
  summarize(n = sum(!is.na(value))) %>% 
  ungroup() %>% 
  mutate(tool_fct = fct_reorder(tool, n)) %>% 
  ggplot(aes(tool_fct, n)) +
    geom_col(aes(tool_fct, 10000, color = tool), fill = "grey90", size = 1.1, width = 0.7) +
    geom_col(aes(fill = tool), color = NA, width = 0.7, alpha = 0.6) +
    geom_text(aes(tool_fct, n - 90, label = comma(n, accuracy = 1)), 
              color = "white", fontface = "bold", family = "Montserrat", size = 2.5, hjust = 1) +
    coord_flip() +
    scale_y_continuous(expand = c(.005, .005),
                       labels = num_format,
                       breaks = seq(0, 10000, by = 2000)) +
    scale_color_carto_d(palette = "Bold", guide = F, limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, limits = levels(df_tools$tool)) +
    labs(x = NULL, y = "Search Requests", subtitle = "Cost per Click (CPC)") +
    theme_flip 

(dat_kws + dat_vol + dat_dif + dat_cpc) * 
  theme(axis.line.y = element_blank(),
        axis.ticks.y = element_blank(),
        axis.text.y = element_text(size = 13, face = "plain"),
        axis.text.x = element_text(size = 8),
        plot.margin = margin(5, 25, 5, 5),
        plot.title = element_markdown(size = 16),
        plot.subtitle = element_text(face = "bold", size = 13)) + 
  plot_layout(nrow = 4, heights = c(.87, 1, 1, 1))

if(save == T){ ggsave(here("plots", "1_DAT_overview_data.png"), width = 8, height = 14, dpi = 300) }

2 Keyword Suggestions

There are 7 SEO tools, each used to search 75 keywords. The keywords fall into 15 categories with each category containing 6. Minimum of suggestions was 22 and the highest values was 6.58880510^{6}.

2.1 The Raw Data On a Logarithmic Scale

To see more of the underlying pattern in keyword suggestions, we transform the x axis to a logarithmic scale and use so-called beeswarm plots to make the distribution more clear to the reader.

Per SEO Tool:

## with log scale
df_kws %>% 
  group_by(tool) %>% 
  mutate(median = median(log10(suggested))) %>% 
  ggplot(aes(fct_reorder(tool, median), suggested)) + 
    stat_summary(fun.y = median, geom = "point", 
                 size = 6, color = "grey70") +
    geom_quasirandom(aes(color = category_fct), 
                     alpha = .7, size = 1.5, 
                     width = .3, bandwidth = .3, varwidth = T) +
    coord_flip() +
    scale_y_log10(labels = num_format) +
    scale_color_simpsons(name = "Keyword Category:") +
    guides(color = guide_legend(override.aes = list(size = 5))) +
    labs(x = NULL, y = "Keyword Suggestions",
         title = "**Each Search by SEO Tool** (logarithmic scale)",
         subtitle = "(tools ranked by overall median)") +
    theme_flip + 
    theme(axis.text.y = element_text(size = 13))

if(save == T){ ggsave(here("plots", "2_1_KWS_bees_tool.png"), width = 14, height = 8, dpi = 300) }

Key takeaways:

• Plot highlights how skewed the data is in terms of keyword categories and SEO tools (with a focus on categories).

• Due to the skewed data, a logarithmic scale helps to visualize the large range of keyword suggestions.

• Google Keyword Planner reaches very low levels of suggestions with a median below 1,000.

• Ahrefs and SEMrush have by far the highest levels of suggestions, followed by Ubersuggest, Moz, and Google Keyword Planner when comparing the overall median.

• Moz returns always a maximum of 1,000 sugestions.
  
  

Per Keyword Category:

## with log scale
df_kws %>% 
  ggplot(aes(category_fct, suggested)) + 
    stat_summary(fun.y = median, geom = "point", 
                 size = 7, color = "grey70") +
    geom_beeswarm(aes(color = tool_fct), 
                      alpha = .7, size = 1.5, cex = .7) +
    coord_flip() +
    scale_y_log10(labels = num_format) +
    scale_color_carto_d(palette = "Prism", name = "SEO Tool:") +
    guides(color = guide_legend(override.aes = list(size = 5))) +
    labs(x = NULL, y = "Keyword Suggestions",
         title = "**Each Search by Keyword Category** (logarithmic scale)",
         subtitle = "(cateogries ranked by overall median)") +
    theme_flip + 
    theme(axis.text.y = element_text(size = 15),
          plot.margin = margin(r = 20))

if(save == T){ ggsave(here("plots", "2_1_KWS_bees_category.png"), width = 14, height = 11, dpi = 300) }

Key takeaways:

• Plot highlights how skewed the data is in terms of keyword categories and SEO tools (with a focus on categories).

• Financial Services, Women Fashion, and Online Marketing gather the most suggestions (ranked 1, 2, and 3 on the y axis), followed by Wedding, Home Improvement, and Vacation Travel when ranked by overall median.

• The distinct number of 1,000 suggestion deliverd by Moz is clearly visible with a blue dotted line at that value.
  
  

2.2 Summary Plots of Distributions

Distributions can be effectively summarized as box and whiskers plots which indicate the median (thick line), the interquartile range (box), the minimum and maximum excluding outliers (whiskers), and outliers (points):

Per SEO Tool:

df_kws %>% 
  ggplot(aes(category_fct, suggested)) + 
    geom_boxplot(aes(color = category_fct), outlier.size = 1) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    coord_flip() +
    scale_y_log10(labels = num_format, 
                  limits = c(range_kw[1], range_kw[2] + 10000000)) +
    scale_color_simpsons(guide = F) +
    labs(x = NULL, y = "Keyword Suggestions",
         title = "**Keyword Categories by SEO Tool**",
         subtitle = "(categories ranked by overall median)") +
    theme_flip +
    theme_facet

if(save == T){ ggsave(here::here("plots", "2_2_KWS_box_facet_tool.png"), width = 14, height = 10, dpi = 300) }

Key takeaways:

• Trends amoung keyword categories are more or less the same besides Google Keyword Planner and Moz which suggest around/always 1,000.

• Interestingly, Financial Services does not score the most suggestions with Google Keyword Planner; Women Fashion and Diet have remarkably higher median values when compared to the other cateories.
  
  

Per Keyword Category:

df_kws %>% 
  ggplot(aes(tool_fct, suggested)) + 
    geom_boxplot(aes(color = tool_fct), outlier.size = 1) +
    facet_wrap(~ category, scales = "free_x", ncol = 5) +
    coord_flip() +
    scale_y_log10(labels = num_format, 
                  limits = c(range_kw[1], range_kw[2] + 10000000)) +
    scale_color_carto_d(palette = "Prism", guide = F) +
    labs(x = NULL, y = "Keyword Suggestions",
         title = "**SEO Tool by Keyword Categories**",
         subtitle = "(tools ranked by overall median)") +
    theme_flip +
    theme_facet

if(save == T){ ggsave(here::here("plots", "2_2_KWS_box_facet_category.png"), width = 14, height = 8, dpi = 300) }

Key takeaways:

• Ahrefs and SEMrush are more competitive in the categories Home Improvement, Legal, Automotive, Solar Energy, Vacations Travel, and Wedding when compared tothe other SEO tools.

• Ahrefs suggest notably more keywords in the categories Office Supplies, Vacations Travel, and Web Hosting

• Ubersuggest yields almost the same median of keyword suggestions as Ahrefs and SEMrush when keywords were belonging to the categories Online Marketing, Vacations Travel, and Diet.

• Google Keyword Planner and Moz reach comparable amounts of suggestions as the other tools only in the categories Web Hosting and Office Supplies.
  
  

The pattern might be easier to read as linerange plot by showing the minimum-maximum range (line) and the median (dot):

Per SEO Tool:

df_kws %>% 
  group_by(category, tool) %>% 
  mutate(
    min = min(suggested),
    max = max(suggested)
  ) %>% 
  ungroup() %>% 
  ggplot(aes(category_fct, suggested, color = category_fct)) + 
    geom_linerange(aes(ymin = min, ymax = max)) +
    stat_summary(fun.y = median, geom = "point", size = 3) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    coord_flip() +
    scale_y_log10(labels = num_format, 
                  limits = c(range_kw[1], range_kw[2] + 10000000)) +
    scale_color_simpsons(guide = F) +
    labs(x = NULL, y = "Keyword Suggestions",
         title = "**Keyword Categories by SEO Tool**",
         subtitle = "(categories ranked by overall median)") +
    theme_flip +
    theme_facet

if(save == T){ ggsave(here::here("plots", "2_2_KWS_range_facet_tool.png"), width = 14, height = 8, dpi = 300) }

Per Keyword Category:

df_kws %>% 
  group_by(category, tool) %>% 
  mutate(
    min = min(suggested),
    max = max(suggested)
  ) %>% 
  ungroup() %>% 
  ggplot(aes(tool_fct, suggested, color = tool_fct)) + 
    geom_linerange(aes(ymin = min, ymax = max)) +
    stat_summary(fun.y = median, geom = "point", size = 3) +
    facet_wrap(~ category, scales = "free_x", ncol = 5) +
    scale_y_log10(labels = num_format, 
                  limits = c(range_kw[1], range_kw[2] + 10000000)) +
    scale_color_carto_d(palette = "Prism", guide = F) +coord_flip() +
    labs(x = NULL, y = "Keyword Suggestions",
         title = "**Keyword Categories by SEO Tool**",
         subtitle = "(tools ranked by overall median)") +
    theme_flip +
    theme_facet

if(save == T){ ggsave(here::here("plots", "2_2_KWS_range_facet_category.png"), width = 14, height = 10, dpi = 300) }

Key takeaways:

• 
  
  

2.3 Keywords per Engine & Category

In the following, we investigate the data in a more detailled manner: each keyword, encoded by SEO tool and keyword category.

All 75 Keywords

df_kws %>% 
  group_by(category, keyword) %>% 
  mutate(sum = sum(suggested)) %>% 
  ungroup() %>% 
  ggplot(aes(fct_reorder(tool_fct, -tool_median), 
             fct_reorder(keyword, sum), 
             size = suggested, fill = category_fct)) +
    geom_point(shape = 21, color = "grey20", stroke = .8) +
    scale_x_discrete(position = "top") +
    scale_y_discrete(expand = c(.01, .01)) +
    scale_fill_simpsons(name = "Keyword Category:") +
    scale_size(name = "Keyword Suggestions:", 
               range = c(2, 10),
               labels = num_format,
               breaks = c(500, 5000, 50000, 500000, 5000000)) +
    guides(
      size = guide_legend(override.aes = list(fill = "grey70")),
      fill = guide_legend(override.aes = list(size = 5), reverse = T)
    ) +
    labs(x = NULL, y = NULL,
         title = "**Performance of All Keywords**",
         subtitle = "(tool ranked by overall median, keywords ranked by total suggestions)\n") +
    theme_facet +
    theme(axis.line.x = element_blank(),
          axis.text.x = element_text(size = 13, face = "bold", angle = 30, 
                                     vjust = 0, hjust = 0),
          axis.ticks.x = element_blank(),
          axis.text.y = element_text(size = 12),
          legend.key.height = unit(25, "pt"),
          legend.position = c(1.3, .75),
          plot.margin = margin(r = 200))

if(save == T){ ggsave(here::here("plots", "2_3_KWS_keywords_tool.png"), width = 10, height = 25, dpi = 300) }

Top 25 Keywords

df_kws %>% 
  group_by(category, keyword) %>% 
  mutate(sum = sum(suggested)) %>% 
  arrange(-sum) %>% 
  group_by(tool) %>% 
  mutate(rank = row_number()) %>% 
  group_by(keyword) %>% 
  mutate(rank = min(rank)) %>% 
  ungroup() %>% 
  filter(rank <= 25) %>% 
  ggplot(aes(fct_reorder(tool_fct, -tool_median), 
             fct_reorder(keyword, sum), 
             size = suggested, fill = category_fct)) +
    geom_point(shape = 21, color = "grey20", stroke = .8) +
    scale_x_discrete(position = "top") +
    scale_y_discrete(expand = c(.02, .02)) +
    scale_fill_simpsons(name = "Keyword Category:", limits = levels(df_kws$category_fct)) +
    scale_size(name = "Keyword Suggestions:", 
               range = c(2, 10),
               labels = num_format,
               breaks = c(500, 5000, 50000, 500000, 5000000)) +
    guides(
      size = guide_legend(override.aes = list(fill = "grey70")),
      fill = guide_legend(override.aes = list(size = 5), reverse = T)
    ) +
    labs(x = NULL, y = NULL,
         title = "**Performance of Top 25 Keywords**",
         subtitle = "(tool ranked by overall median, keywords ranked by total suggestions)\n") +
    theme_facet +
    theme(axis.line.x = element_blank(),
          axis.text.x = element_text(size = 13, face = "bold", angle = 30, 
                                     vjust = 0, hjust = 0),
          axis.ticks.x = element_blank(),
          axis.text.y = element_text(size = 12),
          legend.key.height = unit(25, "pt"),
          legend.position = c(1.3, .55),
          plot.margin = margin(r = 200))

if(save == T){ ggsave(here::here("plots", "2_3_KWS_keywords_tool_top.png"), width = 10, height = 12, dpi = 300) }

Key takeaways:

• 
  
  

… on a Logarithmic Scale

df_kws %>% 
  group_by(category, keyword) %>% 
  mutate(sum = sum(suggested)) %>% 
  ungroup() %>% 
  ggplot(aes(fct_reorder(tool_fct, -tool_median), 
             fct_reorder(keyword, sum), 
             size = log2(suggested), fill = category_fct)) +
    geom_point(shape = 21, color = "grey20", stroke = .8) +
    scale_x_discrete(position = "top") +
    scale_y_discrete(expand = c(.01, .01)) +
    scale_fill_simpsons(name = "Keyword Category:") +
    scale_size(name = "Keyword Suggestions:", 
               range = c(1, 10),
               limits = c(log2(10), log2(10000000)),
               labels = c("10", "100", "1,000", "100,000", "10,000,000"),
               breaks = c(log2(10), log2(100), log2(1000), log2(100000), log2(10000000))) +
    guides(
      size = guide_legend(override.aes = list(fill = "grey70")),
      fill = guide_legend(override.aes = list(size = 5), reverse = T)
    ) +
    labs(x = NULL, y = NULL,
         title = "**Performance of All Keywords**",
         subtitle = "(tool ranked by overall median, keywords ranked by total suggestions)\n") +
    theme_facet +
    theme(axis.line.x = element_blank(),
          axis.text.x = element_text(size = 13, face = "bold", angle = 30, 
                                     vjust = 0, hjust = 0),
          axis.ticks.x = element_blank(),
          axis.text.y = element_text(size = 12),
          legend.key.height = unit(25, "pt"),
          legend.position = c(1.3, .8),
          plot.margin = margin(r = 200))

if(save == T){ ggsave(here::here("plots", "2_3_KWS_keywords_tool_log.png"), width = 10, height = 27, dpi = 300) }

Top 25 Keywords

df_kws %>% 
  group_by(category, keyword) %>% 
  mutate(sum = sum(suggested)) %>% 
  arrange(-sum) %>% 
  group_by(tool) %>% 
  mutate(rank = row_number()) %>% 
  group_by(keyword) %>% 
  mutate(rank = min(rank)) %>% 
  ungroup() %>% 
  filter(rank <= 25) %>% 
  ggplot(aes(fct_reorder(tool_fct, -tool_median), 
             fct_reorder(keyword, sum), 
             size = log2(suggested), fill = category_fct)) +
    geom_point(shape = 21, color = "grey20", stroke = .8) +
    scale_x_discrete(position = "top") +
    scale_y_discrete(expand = c(.02, .02)) +
    scale_fill_simpsons(name = "Keyword Category:", limits = levels(df_kws$category_fct)) +
    scale_size(name = "Keyword Suggestions:", 
               range = c(1, 8),
               limits = c(log2(100), log2(10000000)),
               labels = c("100", "1,000", "100,000", "10,000,000"),
               breaks = c(log2(100), log2(1000), log2(100000), log2(10000000))) +
    guides(
      size = guide_legend(override.aes = list(fill = "grey70")),
      fill = guide_legend(override.aes = list(size = 5), reverse = T)
    ) +
    labs(x = NULL, y = NULL,
         title = "**Performance of Top 25 Keywords**",
         subtitle = "(tool ranked by overall median, keywords ranked by total suggestions)\n") +
    theme_facet +
    theme(axis.line.x = element_blank(),
          axis.text.x = element_text(size = 13, face = "bold", angle = 30, 
                                     vjust = 0, hjust = 0),
          axis.ticks.x = element_blank(),
          axis.text.y = element_text(size = 12),
          legend.key.height = unit(25, "pt"),
          legend.position = c(1.3, .55),
          plot.margin = margin(r = 200))

if(save == T){ ggsave(here::here("plots", "2_3_KWS_keywords_tool_top_log.png"), width = 10, height = 12, dpi = 300) }

---

General Notes

• Multiples could be ordered by ranking of SEO tools (currently as in csv) and/or keyword categories (curently alphabetic) as well.

• Median values and/or ranking could be replaced by mean (note: mean may differ between linear and logarithmic scale)
  
  

---

3 Keyword Search Volume

df_vol <-
  df_tools %>% 
  dplyr::select(tool, keyword, volume, volume_cat) %>% 
  filter(!is.na(volume)) %>% 
  group_by(tool) %>% 
  mutate(tool_median = median(volume)) %>% 
  ungroup() %>% 
  mutate(tool_fct = fct_reorder(tool, tool_median))

## range of volume
range_vol <- range(df_vol$volume)

3.1 Distribution of Keyword Search Volume

df_vol %>% 
  ggplot(aes(tool_fct, volume, 
             color = tool,)) + 
    geom_violin(aes(fill = tool), 
                size = .6, alpha = .2, 
                width = 1.05, trim = F) +
    geom_boxplot(size = .6, width = .15, 
                 coef = 0, outlier.color = NA) +
    coord_flip() +
    scale_y_log10(labels = num_format, 
                  limits = c(2, range_vol[2]),
                  expand = c(.03, .03)) +
    scale_color_carto_d(palette = "Bold", guide = F) +
    scale_fill_carto_d(palette = "Bold", guide = F) +
    labs(x = NULL, y = "Search Volume",
         title = "**Search Volumes by SEO Tool**",
         subtitle = "(tools ranked by overall median)") +
    theme_flip +
    theme(#axis.title.x = element_text(size = 16),
          axis.text.x = element_text(size = 12),
          axis.text.y = element_text(size = 16))

if(save == T){ ggsave(here::here("plots", "3_1_VOL_box_violin_tool.png"), width = 14, height = 9, dpi = 300) }

Key takeaways:

• 
  
  

3.2 Distributions per Volume Category

df_vol %>%
  filter(volume_cat != "0-100") %>% 
  ggplot(aes(tool_fct, volume, 
             color = tool,)) + 
    geom_violin(aes(fill = tool),
                size = .6, alpha = .2,
                width = 1.15, trim = F) +
    geom_boxplot(size = .6, width = .15, alpha = .5,
                 coef = 0, outlier.color = NA) +
    facet_wrap(~ volume_cat, scales = "free_x") +
    coord_flip() +
    scale_y_log10(labels = num_format,
                  breaks = c(100, 300, 1000, 3000, 10000, 100000, 1000000),
                  expand = c(.03, .03)) +
    scale_color_carto_d(palette = "Bold", guide = F) +
    scale_fill_carto_d(palette = "Bold", guide = F) +
    labs(x = NULL, y = "Search Volume",
         title = "**Search Volumes by SEO Tool and Volume Category**",
         subtitle = "(tools ranked by overall median)") +
    theme_flip +
    theme_facet + 
    theme(axis.text.x = element_text(size = 12),
          axis.text.y = element_text(size = 16, face = "bold"),
          panel.spacing.x = unit(2, "lines"),
          strip.text = element_text(size = 15, margin = margin(t = 20, b = 20)))

if(save == T){ ggsave(here::here("plots", "3_2_VOL_box_violin_tool_facet.png"), width = 14, height = 8, dpi = 300) }

3.3 Comparison of Keyword Search Volumes to Google Keyword Planner

df_tools_ref <-
  df_tools_long %>% 
  filter(
    tool == "Google Keyword Planner",
    volume_cat != "0-100"
  ) %>% 
  group_by(category, volume_cat) %>% 
  summarize(ref = mean(value, na.rm = T)) %>% 
  left_join(filter(df_tools_long, tool != "Google Keyword Planner")) %>% 
  group_by(tool, category, volume_cat) %>% 
  summarize(difference = mean(value, na.rm = T) - unique(ref)) %>% 
ungroup()

df_tools_ref %>% 
  filter(
    category == "volume",
    volume_cat != "0-100"
  ) %>% 
  mutate(
    tool_fct = fct_reorder(tool, difference),
    pos_r = if_else(difference >= 0, difference, NA_real_),
    pos_l = if_else(difference < 0, difference, NA_real_)
  ) %>% 
  ggplot(aes(tool_fct, difference, 
             color = tool)) + 
    geom_segment(aes(x = tool_fct, xend = tool_fct, 
                     y = 0, yend = difference),
                 size = 1.5) +
    geom_hline(yintercept = 0, color = "grey70", size = .9) +
    geom_point(size = 5) +
    geom_text(aes(tool_fct, pos_r, label = tool_fct), 
              hjust = 0, nudge_y = 1400,
              family = "Montserrat", 
              fontface = "bold", size = 4.2) +
    geom_text(aes(tool_fct, pos_l, label = tool_fct), 
              hjust = 1,  nudge_y = -1400,
              family = "Montserrat", 
              fontface = "bold", size = 4.2) +
    facet_wrap(~ volume_cat) +
    coord_flip() +
    scale_y_continuous(labels = num_format,
                       limits = c(-7000, 30000),
                       expand = c(.3, .3)) +
    scale_color_carto_d(palette = "Bold", 
                        limits = levels(df_tools$tool),
                        guide = F) +
    scale_fill_carto_d(palette = "Bold", 
                       limits = levels(df_tools$tool),
                       guide = F) +
    labs(x = NULL, y = "Difference in Average Search Volume",
         title = "**Comparison of Average Search Volume to Google Keyword Planner**",
         subtitle = "(tools ranked by overall median, subsetted by volume category)") +
    theme_flip + 
    theme_lolli +
    theme(panel.spacing = unit(2, "lines"),
          strip.text = element_text(size = 15, face = "bold", margin = margin(t = 20, b = 20)))

if(save == T){ ggsave(here::here("plots", "3_3_VOL_lollipops_avg_diff_cat.png"), width = 14, height = 5, dpi = 300) }

3.4 Top Keywords per SEO Tool

df_keywords_top <-
  df_tools_long %>%
  group_by(tool, category) %>% 
  arrange(-value) %>% 
  mutate(
    rank = row_number(),
    max_rank = max(rank)
  ) %>% 
  filter(rank <= 10) %>% 
  ungroup()

theme_heat <-
  theme(axis.line.x = element_blank(),
        axis.line.y = element_blank(),
        axis.text.x = element_text(size = 12, angle = 30, 
                                   vjust = 0, hjust = 0),
        axis.text.y = element_text(size = 11, hjust = 0),
        axis.ticks = element_blank(),
        plot.margin = margin(5.5, 22, 5.5, 5.5))

## heatmap top volume
df_keywords_top %>% 
  filter(
    category == "volume",
    !is.na(value)
  ) %>% 
  mutate(
    mio = round(value / 1000000, 2),
    value_log = log10(value),
    font_col = if_else(value <= max(value) - ((max(value) - min(value)) / 2), "light", "dark"),
    label = glue::glue("{str_wrap(keyword, 15)}\n{mio}M")
  ) %>% 
  #mutate(keyword = fct_reorder(keyword, desc(rank_total))) %>% 
  ggplot(aes(tool, rank, fill = value_log, label = label)) +
    geom_tile(color = "white", size = .5) +
    geom_text(aes(color = font_col),
              family = "Montserrat", fontface = "bold", 
              size = 3.3, lineheight = .95) +
    scale_x_discrete(expand = c(0, 0), position = "top") +
    scale_y_reverse(expand = c(0, 0), breaks = 1:10, 
                    labels = glue::glue("Rank {1:10}" )) +
    scale_color_manual(values = c('dark' = 'black', 'light' = 'white'), 
                       guide = F) +
    scico::scale_fill_scico(palette = "lapaz", guide = F) +
    labs(x = NULL, y = NULL,
         title = "**Top 10 Keywords of Each SEO Tool** (with Regard to Search Volume)") +
    theme_heat

if(save == T){ ggsave(here::here("plots", "3_4_VOL_heatmap_keywords_top.png"), width = 14, height = 8, dpi = 300) }

Key takeaways:

• 
  
  

3.5 Keywords by Length

df_vol %>% 
  mutate(length = nchar(keyword)) %>% 
  ggplot(aes(length, volume, color = tool, fill = tool)) +
    geom_jitter(width = .2, alpha = .1, size = .5) +
    geom_smooth(method = "lm", color = "grey7", size = .4, alpha = .4) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    scale_x_continuous(limits = c(1, 60)) +
    scale_y_log10(labels = num_format, expand = c(.1, .1)) +
    scale_color_carto_d(palette = "Bold", guide = F) +
    scale_fill_carto_d(palette = "Bold", guide = F) +
    labs(x = "Number of Keyword Characters", 
         y = "Keyword Suggestions",
         title = "**Search Volume in Relation to Keyword Length per SEO Tool**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25))

if(save == T){ ggsave(here::here("plots", "3_5_VOL_length_tool_lm.png"), width = 14, height = 6.5, dpi = 300) }

Key takeaways:

• 
  
  

---

4 Keyword Difficulty Scores

df_diff <-
  df_tools %>% 
  dplyr::select(tool, keyword, diff, volume_cat) %>% 
  filter(!is.na(diff)) %>% 
  group_by(tool) %>% 
  mutate(tool_median = median(diff)) %>% 
  ungroup() %>% 
  mutate(tool_fct = fct_reorder(tool, tool_median))

## range of difficulty
range_diff <- range(df_diff$diff)

4.1 Distribution of Difficulty Scores

df_diff %>% 
  ggplot(aes(tool_fct, diff, 
             color = tool)) + 
    #geom_violin(aes(fill = tool), 
    #            alpha = .3, width = 1.4, trim = F) +
    #geom_boxplot(width = .1, coef = 0, outlier.color = NA) +
    geom_boxplot(width = .7, size = 1, alpha = .3, outlier.size = .8) +
    coord_flip() +
    scale_y_log10(labels = num_format, 
                  limits = c(.9, range_diff[2])) +
    scale_color_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    labs(x = NULL, y = "Difficulty Score",
         title = "**Difficulty Scores  by SEO Tool**",
         subtitle = "(tools ranked by overall median)") +
    theme_flip +
    theme(axis.text.x = element_text(size = 12),
          axis.text.y = element_text(size = 16))

if(save == T){ ggsave(here::here("plots", "4_1_DIF_box_tool.png"), width = 14, height = 8, dpi = 300) }

Key takeaways:

• 
  
  

4.2 Distributions of Difficulty Scores by Volume Categories

df_diff %>% 
  filter(volume_cat != "0-100") %>% 
  ggplot(aes(tool_fct, diff, 
             color = tool)) + 
    #geom_violin(aes(fill = tool), 
    #            alpha = .3, width = 1.4, trim = F) +
    #geom_boxplot(width = .1, coef = 0, outlier.color = NA) +
    geom_boxplot(width = .7, size = 1, alpha = .3, outlier.size = .8) +
    facet_wrap(~ volume_cat) + 
    coord_flip() +
    scale_y_log10(labels = num_format, 
                  limits = c(.9, range_diff[2])) +
    scale_color_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    labs(x = NULL, y = "Difficulty Score",
         title = "**Difficulty Scores  by SEO Tool and Volume Category**",
         subtitle = "(tools ranked by overall median)") +
    theme_flip +
    theme_facet + 
    theme(axis.text.x = element_text(size = 12),
          axis.text.y = element_text(size = 16, face = "bold"),
          panel.spacing.x = unit(2, "lines"),
          strip.text = element_text(size = 15, margin = margin(t = 20, b = 20)))

if(save == T){ ggsave(here::here("plots", "4_2_DIF_box_tool_cat.png"), width = 14, height = 7, dpi = 300) }

Key takeaways:

• 
  
  

4.3 Top Keywords per SEO Tool

## heatmap top difficulty
df_keywords_top %>% 
  filter(
    category == "diff",
    !is.na(value)
  ) %>% 
  mutate(
    value_log = log10(value),
    font_col = if_else(value <= max(value) - ((max(value) - min(value)) / 2), "light", "dark"),
    label = glue::glue("{str_wrap(keyword, 20)}\n{round(value, 1)}%")
  ) %>% 
  ggplot(aes(tool, rank, fill = value_log, label = label)) +
    geom_tile(color = "white", size = .5) +
    geom_text(aes(color = font_col),
              family = "Montserrat", fontface = "bold", 
              size = 3.5, lineheight = .95) +
    scale_x_discrete(expand = c(0, 0), position = "top") +
    scale_y_reverse(expand = c(0, 0), breaks = 1:10, 
                    labels = glue::glue("Rank {1:10}" )) +
    scale_color_manual(values = c('dark' = 'black', 'light' = 'white'), 
                       guide = F) +
    scico::scale_fill_scico(palette = "lapaz", guide = F) +
    labs(x = NULL, y = NULL,
         title = "**Top 10 Keywords of Each SEO Tool** (with Regard to Difficulty Scores)") +
    theme_heat

if(save == T){ ggsave(here::here("plots", "4_3_DIF_heatmap_keywords_top.png"), width = 14, height = 8, dpi = 300) }

# ## heatmap bottom volume
# df_keywords_top %>% 
#   filter(category == "volume", rank < 0) %>% 
#   mutate(
#     value = log10(value),
#     max = max(value, na.rm = T),
#     min = min(value, na.rm = T),
#     font_col = if_else(value <= max(value) - ((max(value) - min(value)) / 2), "light", "dark"),
#     label = glue::glue("<b>{keyword}</b><br><span style='font-size:9pt'>{round(value, 2)}M</span>")
#   ) %>% 
#   #mutate(keyword = fct_reorder(keyword, desc(rank_total))) %>% 
#   ggplot(aes(tool, rank, fill = value, label = str_wrap(label))) +
#     geom_tile(color = "white", size = 0.5) +
#     geom_richtext(aes(color = font_col),
#                   family = "Montserrat", size = 3,
#                   fill = NA, label.color = NA) +
#     scale_x_discrete(expand = c(0, 0), position = "top") +
#     scale_y_continuous(expand = c(0, 0), breaks = -1:-10, 
#                     labels = glue::glue("Rank {1:10}")) +
#     scale_color_manual(values = c('dark' = 'black', 'light' = 'white'), 
#                        guide = F) +
#     scico::scale_fill_scico(palette = "turku", guide = F) +
#     labs(x = NULL, y = NULL) +
#     theme_heat

Key takeaways:

• 
  
  

5 Cost per Click (CPC)

df_cpc <-
  df_tools %>% 
  dplyr::select(tool, keyword, cpc, volume_cat) %>% 
  filter(!is.na(cpc)) %>% 
  group_by(tool) %>% 
  mutate(tool_median = median(cpc)) %>% 
  ungroup() %>% 
  mutate(tool_fct = fct_reorder(tool, tool_median))

## range of cpc
range_cpc <- range(df_cpc$cpc)

5.1 Distribution of CPC per SEO Tool

df_cpc %>% 
  ggplot(aes(cpc)) +
    geom_histogram(aes(color = tool, fill = tool), binwidth = .2) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    coord_cartesian(xlim = c(.005, range_cpc[2] + 500)) +
    scale_x_log10(labels = num_format) +
    scale_color_carto_d(palette = "Bold", guide = F, 
                        limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    labs(x = "CPC", y = "Count",
         title = "**Distribution of Cost per Click (CPC) by SEO Tool**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "5_1_CPC_histo_log_tool.png"), width = 14, height = 7, dpi = 300) }

Key takeaways:

• 
  
  

df_cpc %>% 
  ggplot(aes(tool_fct, cpc, 
             color = tool)) + 
    #geom_violin(aes(fill = tool), 
    #            alpha = .3, width = 1.5, trim = F) +
    #geom_boxplot(width = .1, coef = 0, outlier.color = NA) +
    geom_boxplot(width = .7, size = 1, alpha = .3, outlier.size = .8) +
    coord_flip() +
    scale_y_log10(labels = num_format, 
                  limits = c(.01, range_cpc[2])) +
    scale_color_carto_d(palette = "Bold", guide = F, 
                        limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    labs(x = NULL, y = "CPC",
         title = "**Cost per Click (CPC) by SEO Tool**",
         subtitle = "(tools ranked by overall median)") +
    theme_flip +
    theme(axis.text.x = element_text(size = 12),
          axis.text.y = element_text(size = 16))

if(save == T){ ggsave(here::here("plots", "5_1_CPC_box_tool.png"), width = 14, height = 9, dpi = 300) }

Key takeaways:

• CPC are almost the same for all tools with SECockpit having the highest median, folloed by Google Keyword Planner. SEMrush has the slightly lowest median.
  
  

5.2 Distributions of CPC per Volume Category

df_cpc %>% 
  filter(volume_cat != "0-100") %>% 
  ggplot(aes(tool_fct, cpc, 
             color = tool)) + 
    #geom_violin(aes(fill = tool), 
    #            alpha = .3, width = 1.5, trim = F) +
    #geom_boxplot(width = .1, coef = 0, outlier.color = NA) +
    geom_boxplot(width = .7, size = 1, alpha = .3, outlier.size = .8) +
    facet_wrap(~ volume_cat) +
    coord_flip() +
    scale_y_log10(labels = num_format, 
                  limits = c(.01, range_cpc[2] + 500)) +
    scale_color_carto_d(palette = "Bold", guide = F, 
                        limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    labs(x = NULL, y = "CPC",
         title = "**Cost per Click (CPC) by SEO Tool and Volume Category**",
         subtitle = "(tools ranked by overall median)") +
    theme_flip +
    theme_facet + 
    theme(axis.text.x = element_text(size = 12),
          axis.text.y = element_text(size = 16, face = "bold"),
          panel.spacing.x = unit(2, "lines"),
          strip.text = element_text(size = 15, margin = margin(t = 20, b = 20)))

if(save == T){ ggsave(here::here("plots", "5_2_CPC_box_tool_cat.png"), width = 14, height = 8, dpi = 300) }

Key takeaways:

• Patterns are almost the same independent from SEO tool and volume category.
  
  

5.3 Comparison of Keyword Search Volumes to Google Keyword Planner

df_tools_ref %>% 
  filter(
    category == "cpc",
    volume_cat != "0-100"
  ) %>% 
  mutate(
    tool_fct = fct_reorder(tool, difference),
    pos_r = if_else(difference >= 0, difference, NA_real_),
    pos_l = if_else(difference < 0, difference, NA_real_)
  ) %>% 
  ggplot(aes(tool_fct, difference, 
             color = tool)) + 
    geom_segment(aes(x = tool_fct, xend = tool_fct, 
                     y = 0, yend = difference),
                 size = 1.5) +
    geom_hline(yintercept = 0, color = "grey70", size = .9) +
    geom_point(size = 5) +
    geom_text(aes(tool_fct, pos_r, label = tool_fct), 
              hjust = 0, nudge_y = .15,
              family = "Montserrat", 
              fontface = "bold", size = 4.2) +
    geom_text(aes(tool_fct, pos_l, label = tool_fct), 
              hjust = 1,  nudge_y = -.15,
              family = "Montserrat", 
              fontface = "bold", size = 4.2) +
    facet_wrap(~ volume_cat) +
    coord_flip() +
    scale_y_continuous(labels = num_format,
                       limits = c(-2.5, 1),
                       expand = c(.2, .2)) +
    scale_color_carto_d(palette = "Bold", 
                        limits = levels(df_tools$tool),
                        guide = F) +
    scale_fill_carto_d(palette = "Bold", 
                       limits = levels(df_tools$tool),
                       guide = F) +
    labs(x = NULL, y = "Difference in Average Cost per Click (CPC)",
         title = "**Comparison of Average Cost per Click (CPC) to Google Keyword Planner**",
         subtitle = "(tools ranked by overall median, subsetted by volume category)") +
    theme_flip + 
    theme_lolli +
    theme(panel.spacing = unit(2, "lines"),
          strip.text = element_text(size = 15, face = "bold", margin = margin(t = 20, b = 20)))

if(save == T){ ggsave(here::here("plots", "5_3_CPC_lollipops_avg_diff_cat.png"), width = 14, height = 5, dpi = 300) }

5.4 Top Keywords per SEO Tool

## heatmap top difficulty
df_keywords_top %>% 
  filter(
    category == "cpc",
    !is.na(value)
  ) %>% 
  mutate(
    value_log = log10(value),
    font_col = if_else(value <= max(value) - ((max(value) - min(value)) / 2), "light", "dark"),
    label = glue::glue("{str_wrap(keyword, 18)}\n${round(value, 0)}")
  ) %>% 
  ggplot(aes(tool, rank, fill = value_log, label = label)) +
    geom_tile(color = "white", size = .5) +
    geom_text(aes(color = font_col),
              family = "Montserrat", fontface = "bold", 
              size = 3.3, lineheight = .95) +
    scale_x_discrete(expand = c(0, 0), position = "top") +
    scale_y_reverse(expand = c(0, 0), breaks = 1:10, 
                    labels = glue::glue("Rank {1:10}" )) +
    scale_color_manual(values = c('dark' = 'black', 'light' = 'white'), guide = F) +
    scico::scale_fill_scico(palette = "lapaz", guide = F) +
    labs(x = NULL, y = NULL,
         title = "**Top 10 Keywords of Each SEO Tool** (with regard to Cost per Click)") +
    theme_heat

if(save == T){ ggsave(here::here("plots", "5_4_CPC_heatmap_keywordstop.png"), width = 14, height = 8, dpi = 300) }

Key takeaways:

• 
  
  

6. Performance in Volume, Difficulty & CPC

6.1 Heatmap of Average Performance

df_tools %>% 
  dplyr::select(-tool_id) %>% 
  pivot_longer(
    c(volume, diff, cpc), 
    names_to = "category", 
    values_to = "value"
  ) %>% 
  group_by(category) %>% 
  mutate(
    avg = mean(value, na.rm = T),
    comp = value / avg
  ) %>% 
  group_by(tool, category) %>% 
  summarize(
    avg = mean(comp, na.rm = T),
    log_avg = log2(avg),
    sd = sd(comp, na.rm = T)
  ) %>% 
  ggplot(aes(tool, category, fill = log_avg)) +
    geom_tile(color = "white", size = 0.8) +
    coord_equal() +
    scale_x_discrete(position = "top", 
                     expand = c(0, 0)) +
    scale_y_discrete(labels = c("Cost per Click (CPC)", "Difficulty Scores", "Search Volume"),
                     expand = c(0, 0)) +
    scico::scale_fill_scico(palette = "cork",
                            name = "Difference to Overall Average",
                            limits = c(-1.2, 1.2),
                            breaks = seq(-1, 1, by = 1),
                            labels = c("50%", "100%", "200%")) +
    labs(x = NULL, y = NULL,
         title = "**Comparison of Performance in Volume, Difficulty & CPC**<br>") +
    guides(fill = guide_colorbar(barheight = unit(3, units = "mm"),  
                                 barwidth = unit(100, units = "mm"),
                                 direction = "horizontal",
                                 ticks.colour = "white",
                                 title.position = "top",
                                 title.hjust = 0.5)) +
    theme(axis.line.x = element_blank(),
          axis.line.y = element_blank(),
          axis.text.x = element_text(size = 12, face = "bold", angle = 30, 
                                     vjust = 0, hjust = 0),
          axis.text.y = element_text(size = 12, face = "bold"),
          axis.ticks = element_blank(), 
          legend.position = "bottom",
          legend.title = element_text(size = 11),
          legend.text = element_text(size = 9),
          plot.margin = margin(5.5, 22, 5.5, 5.5))

if(save == T){ ggsave(here::here("plots", "6_1_ALL_heatmap_tool.png"), width = 10, height = 5.7, dpi = 300) }

Key takeaways:

• 
  
  

6.2 Engine Performance Compared to Google Keyword Planner

Lollipop Plots of Average Difference

## comparison of average vol, diff and cpc
df_tools_ref <-
  df_tools_long %>% 
  filter(tool == "Google Keyword Planner") %>% 
  group_by(category) %>% 
  summarize(ref = mean(value, na.rm = T)) %>% 
  left_join(filter(df_tools_long, tool != "Google Keyword Planner")) %>% 
  group_by(tool, category) %>% 
  summarize(difference = mean(value, na.rm = T) - unique(ref)) %>% 
ungroup()

## plot search volume
lolli_vol <-
  df_tools_ref %>% 
  filter(category == "volume") %>% 
  mutate(
    tool_fct = fct_reorder(tool, difference),
    pos_r = if_else(difference >= 0, difference, NA_real_),
    pos_l = if_else(difference < 0, difference, NA_real_)
  ) %>% 
  ggplot(aes(tool_fct, difference, 
             color = tool)) + 
    geom_segment(aes(x = tool_fct, xend = tool_fct, 
                     y = 0, yend = difference),
                 size = 1.5) +
    geom_hline(yintercept = 0, color = "grey70", size = .9) +
    geom_point(size = 5) +
    geom_text(aes(tool_fct, pos_r, label = tool_fct), 
              hjust = 0, nudge_y = 600,
              family = "Montserrat", 
              fontface = "bold", size = 4.7) +
    geom_text(aes(tool_fct, pos_l, label = tool_fct), 
              hjust = 1,  nudge_y = -600,
              family = "Montserrat", 
              fontface = "bold", size = 4.7) +
    coord_flip() +
    scale_y_continuous(labels = num_format,
                       limits = c(-7000, NA),
                       expand = c(.3, .3)) +
    scale_color_carto_d(palette = "Bold", 
                        limits = levels(df_tools$tool),
                        guide = F) +
    scale_fill_carto_d(palette = "Bold", 
                       limits = levels(df_tools$tool),
                       guide = F) +
    labs(x = NULL, y = "Difference in Average Search Volume") +
    theme_flip + 
    theme_lolli

## plot difficulty scores
## not useful anymore since no difficulty output for GKP
# lolli_dif <-
#   df_tools_ref %>% 
#   filter(category == "diff") %>% 
#   mutate(
#     tool_fct = fct_reorder(tool, difference),
#     pos_r = if_else(difference >= 0, difference, NA_real_),
#     pos_l = if_else(difference < 0, difference, NA_real_)
#   ) %>% 
#   ggplot(aes(tool_fct, difference, 
#              color = tool)) + 
#     geom_segment(aes(x = tool_fct, xend = tool_fct, 
#                      y = 0, yend = difference),
#                  size = 1.5) +
#     geom_hline(yintercept = 0, color = "grey70", size = .9) +
#     geom_point(size = 5) +
#     geom_text(aes(tool_fct, pos_r, label = tool_fct), 
#               hjust = 0, nudge_y = 3,
#               family = "Montserrat", 
#               fontface = "bold", size = 4.7) +
#     geom_text(aes(tool_fct, pos_l, label = tool_fct), 
#               hjust = 1,  nudge_y = -3,
#               family = "Montserrat", 
#               fontface = "bold", size = 4.7) +
#     coord_flip() +
#     scale_y_continuous(labels = num_format,
#                        expand = c(.2, .2)) +
#     scale_color_carto_d(palette = "Bold", 
#                         limits = levels(df_tools$tool),
#                         guide = F) +
#     scale_fill_carto_d(palette = "Bold", 
#                        limits = levels(df_tools$tool),
#                        guide = F) +
#     labs(x = NULL, y = "Difference in Average Difficulty Score",
#          title = "**Performance of SEO Tools in Comparison to Google Keyword Planner** (comparison of averages)") +
#     theme_flip + 
#     theme_lolli +
#     theme(plot.title = element_markdown(size = 22, margin = margin(b = 20)))

## plot cost per click
lolli_cpc <-
  df_tools_ref %>% 
  filter(category == "cpc") %>% 
  mutate(
    tool_fct = fct_reorder(tool, difference),
    pos_r = if_else(difference >= 0, difference, NA_real_),
    pos_l = if_else(difference < 0, difference, NA_real_)
  ) %>% 
  ggplot(aes(tool_fct, difference, 
             color = tool)) + 
    geom_segment(aes(x = tool_fct, xend = tool_fct, 
                     y = 0, yend = difference),
                 size = 1.5) +
    geom_hline(yintercept = 0, color = "grey70", size = .9) +
    geom_point(size = 5) +
    geom_text(aes(tool_fct, pos_r, label = tool_fct), 
              hjust = 0, nudge_y = .1,
              family = "Montserrat", 
              fontface = "bold", size = 4.7) +
    geom_text(aes(tool_fct, pos_l, label = tool_fct), 
              hjust = 1,  nudge_y = -.1,
              family = "Montserrat", 
              fontface = "bold", size = 4.7) +
    coord_flip() +
    scale_y_continuous(labels = num_format,
                       limits = c(-2.1, .5),
                       expand = c(.2, .2)) +
    scale_color_carto_d(palette = "Bold", 
                        limits = levels(df_tools$tool),
                        guide = F) +
    scale_fill_carto_d(palette = "Bold", 
                       limits = levels(df_tools$tool),
                       guide = F) +
    labs(x = NULL, y = "Difference in Average Cost per Click (CPC)") +
    theme_flip + 
    theme_lolli

## centered main title
title <- ggplot(data.frame(x = 1:2, y = 1:10)) +
  labs(x = NULL, y = NULL,
       title = "**Performance of SEO Tools in Comparison to Google Keyword Planner**",
       subtitle = "(comparison of averages)") +
  theme(line = element_blank(),
        plot.background = element_rect(fill = NA, color = NA),
        panel.background = element_rect(fill = NA, color = NA),
        axis.text = element_blank(),
        axis.line.x = element_blank(),
        axis.line.y = element_blank())

title + (lolli_vol + lolli_cpc) + plot_layout(heights = c(0, 1))

if(save == T){ ggsave(here::here("plots", "6_2_ALL_lollipops_vol_cpc_average.png"), width = 14, height = 5, dpi = 300) }

Key takeaways:

• 
  
  

Lollipop Plots of Median Difference

## comparison of median vol, diff and cpc
df_tools_ref_m <-
  df_tools_long %>% 
  filter(tool == "Google Keyword Planner") %>% 
  group_by(category) %>% 
  summarize(ref = median(value, na.rm = T)) %>% 
  left_join(filter(df_tools_long, tool != "Google Keyword Planner")) %>% 
  group_by(tool, category) %>% 
  summarize(difference = median(value, na.rm = T) - unique(ref)) %>% 
  ungroup()

## plot search volume
lolli_vol_m <-
  df_tools_ref_m %>% 
  filter(category == "volume") %>% 
  mutate(
    tool_fct = fct_reorder(tool, difference),
    pos_r = if_else(difference >= 0, difference, NA_real_),
    pos_l = if_else(difference < 0, difference, NA_real_)
  ) %>% 
  ggplot(aes(tool_fct, difference, 
             color = tool)) + 
    geom_segment(aes(x = tool_fct, xend = tool_fct, 
                     y = 0, yend = difference),
                 size = 1.5) +
    geom_hline(yintercept = 0, color = "grey70", size = .9) +
    geom_point(size = 5) +
    geom_text(aes(tool_fct, pos_r, label = tool_fct), 
              hjust = 0, nudge_y = 400,
              family = "Montserrat", 
              fontface = "bold", size = 4.7) +
    geom_text(aes(tool_fct, pos_l, label = tool_fct), 
              hjust = 1,  nudge_y = -400,
              family = "Montserrat", 
              fontface = "bold", size = 4.7) +
    coord_flip() +
    scale_y_continuous(labels = num_format,
                       limits = c(-3000, NA),
                       expand = c(.15, .15)) +
    scale_color_carto_d(palette = "Bold", 
                        limits = levels(df_tools$tool),
                        guide = F) +
    scale_fill_carto_d(palette = "Bold", 
                       limits = levels(df_tools$tool),
                       guide = F) +
    labs(x = NULL, y = "Difference in Median Search Volume") +
    theme_flip + 
    theme_lolli

## plot difficulty scores
## not useful anymore since no difficulty output for GKP
# lolli_dif_m <-
#   df_tools_ref_m %>% 
#   filter(category == "diff") %>% 
#   mutate(
#     tool_fct = fct_reorder(tool, difference),
#     pos_r = if_else(difference >= 0, difference, NA_real_),
#     pos_l = if_else(difference < 0, difference, NA_real_)
#   ) %>% 
#   ggplot(aes(tool_fct, difference, 
#              color = tool)) + 
#     geom_segment(aes(x = tool_fct, xend = tool_fct, 
#                      y = 0, yend = difference),
#                  size = 1.5) +
#     geom_hline(yintercept = 0, color = "grey70", size = .9) +
#     geom_point(size = 5) +
#     geom_text(aes(tool_fct, pos_r, label = tool_fct), 
#               hjust = 0, nudge_y = 3,
#               family = "Montserrat", 
#               fontface = "bold", size = 4.7) +
#     geom_text(aes(tool_fct, pos_l, label = tool_fct), 
#               hjust = 1,  nudge_y = -3,
#               family = "Montserrat", 
#               fontface = "bold", size = 4.7) +
#     coord_flip() +
#     scale_y_continuous(labels = num_format,
#                        limits = c(-80, 45),
#                        expand = c(.1, .1)) +
#     scale_color_carto_d(palette = "Bold", 
#                         limits = levels(df_tools$tool),
#                         guide = F) +
#     scale_fill_carto_d(palette = "Bold", 
#                        limits = levels(df_tools$tool),
#                        guide = F) +
#     labs(x = NULL, y = "Difference in Median Difficulty Score",
#          title = "**Performance of SEO Tools in Comparison to Google Keyword Planner** (comparison of medians)") +
#     theme_flip + 
#     theme_lolli +
#     theme(plot.title = element_markdown(size = 22, margin = margin(b = 20)))

## plot cost per click
lolli_cpc_m <-
  df_tools_ref_m %>% 
  filter(category == "cpc") %>% 
  mutate(
    tool_fct = fct_reorder(tool, difference),
    pos_r = if_else(difference >= 0, difference, NA_real_),
    pos_l = if_else(difference < 0, difference, NA_real_)
  ) %>% 
  ggplot(aes(tool_fct, difference, 
             color = tool)) + 
    geom_segment(aes(x = tool_fct, xend = tool_fct, 
                     y = 0, yend = difference),
                 size = 1.5) +
    geom_hline(yintercept = 0, color = "grey70", size = .9) +
    geom_point(size = 5) +
    geom_text(aes(tool_fct, pos_r, label = tool_fct), 
              hjust = 0, nudge_y = .08,
              family = "Montserrat", 
              fontface = "bold", size = 4.7) +
    geom_text(aes(tool_fct, pos_l, label = tool_fct), 
              hjust = 1,  nudge_y = -.08,
              family = "Montserrat", 
              fontface = "bold", size = 4.7) +
    coord_flip() +
    scale_y_continuous(labels = num_format,
                       limits = c(-.9, .3),
                       expand = c(.2, .2)) +
    scale_color_carto_d(palette = "Bold", 
                        limits = levels(df_tools$tool),
                        guide = F) +
    scale_fill_carto_d(palette = "Bold", 
                       limits = levels(df_tools$tool),
                       guide = F) +
    labs(x = NULL, y = "Difference in Median Cost per Click (CPC)") +
    theme_flip + 
    theme_lolli

## centered main title
title_m <- ggplot(data.frame(x = 1:2, y = 1:10)) +
  labs(x = NULL, y = NULL,
       title = "**Performance of SEO Tools in Comparison to Google Keyword Planner**",
       subtitle = "(comparison of medians)") +
  theme(line = element_blank(),
        plot.background = element_rect(fill = NA, color = NA),
        panel.background = element_rect(fill = NA, color = NA),
        axis.text = element_blank(),
        axis.line.x = element_blank(),
        axis.line.y = element_blank())

title_m + (lolli_vol_m + lolli_cpc_m) + plot_layout(heights = c(0, 1))

if(save == T){ ggsave(here::here("plots", "6_2_ALL_lollipops_tool_median.png"), width = 14, height = 5, dpi = 300) }

Key takeaways:

• 
  
  

6.2 Heatmap

df_keywords_top <-
  df_tools_long %>% 
  group_by(category, keyword) %>% 
  summarize(avg = mean(value, na.rm = T)) %>% 
  group_by(category) %>% 
  arrange(-avg) %>% 
  mutate(rank_total = row_number()) %>% 
  filter(rank_total <= 10 | rank_total > (max(rank_total) - 10))

df_keywords_rank <-
  df_tools_long %>% 
  inner_join(df_keywords_top) %>% 
  #filter(keyword %in% df_keywords_top$keyword) %>% 
  group_by(category) %>% 
  arrange(-value) %>% 
  mutate(rank_tool = row_number())

## heatmap volume
df_keywords_rank %>% 
  filter(category == "volume", rank_total <= 10) %>% 
  mutate(keyword = fct_reorder(keyword, desc(rank_total))) %>% 
  ggplot(aes(tool, keyword, fill = value)) +
    geom_tile()

---

Appendix

Additional Tables & Visualizations

2 Keyword Suggestions

Data Table

## table
df_kws %>%
  dplyr::select(
    Category = "category",
    Keyword = "keyword",
    `SEO Tool` = "tool",
    Suggestions = "suggested",
    `Median per Tool` = "tool_median",
    `Median per Category` = "category_median"
  ) %>% 
  kable(format.args = list(big.mark = ",", 
                           small.mark = ",", 
                           decimal.mark = ".", 
                           scientific = F)) %>%
  kable_styling(fixed_thead = T) %>%
  collapse_rows(columns = 1:2, valign = "top") %>%
  scroll_box(width = "100%", height = "600px")

Category	Keyword	SEO Tool	Suggestions	Median per Tool	Median per Category
Financial Services	finance	Google Keyword Planner	691	691	153,308.0
		Ahrefs (all keyword ideas)	1,269,225	43,488	153,308.0
		Ahrefs (phrase match)	1,233,201	23,021	153,308.0
		SEMrush (broad match)	1,201,273	36,523	153,308.0
		SEMrush (phrase match)	973,555	25,475	153,308.0
		Moz (mix of sources)	1,000	1,000	153,308.0
		Ubersuggest (related)	196,562	4,261	153,308.0
	investing	Google Keyword Planner	2,528	691	153,308.0
		Ahrefs (all keyword ideas)	264,668	43,488	153,308.0
		Ahrefs (phrase match)	255,608	23,021	153,308.0
		SEMrush (broad match)	1,209,214	36,523	153,308.0
		SEMrush (phrase match)	135,075	25,475	153,308.0
		Moz (mix of sources)	1,000	1,000	153,308.0
		Ubersuggest (related)	29,295	4,261	153,308.0
	loans	Google Keyword Planner	936	691	153,308.0
		Ahrefs (all keyword ideas)	1,007,861	43,488	153,308.0
		Ahrefs (phrase match)	994,674	23,021	153,308.0
		SEMrush (broad match)	1,851,628	36,523	153,308.0
		SEMrush (phrase match)	541,163	25,475	153,308.0
		Moz (mix of sources)	1,000	1,000	153,308.0
		Ubersuggest (related)	153,308	4,261	153,308.0
	insurance	Google Keyword Planner	789	691	153,308.0
		Ahrefs (all keyword ideas)	6,588,805	43,488	153,308.0
		Ahrefs (phrase match)	6,522,620	23,021	153,308.0
		SEMrush (broad match)	4,109,928	36,523	153,308.0
		SEMrush (phrase match)	4,003,105	25,475	153,308.0
		Moz (mix of sources)	1,000	1,000	153,308.0
		Ubersuggest (related)	823,235	4,261	153,308.0
	financial advisor	Google Keyword Planner	695	691	153,308.0
		Ahrefs (all keyword ideas)	74,293	43,488	153,308.0
		Ahrefs (phrase match)	68,696	23,021	153,308.0
		SEMrush (broad match)	59,695	36,523	153,308.0
		SEMrush (phrase match)	41,971	25,475	153,308.0
		Moz (mix of sources)	1,000	1,000	153,308.0
		Ubersuggest (related)	9,373	4,261	153,308.0
Diet	how to lose weight	Google Keyword Planner	1,648	691	10,055.0
		Ahrefs (all keyword ideas)	92,045	43,488	10,055.0
		Ahrefs (phrase match)	30,342	23,021	10,055.0
		SEMrush (broad match)	45,852	36,523	10,055.0
		SEMrush (phrase match)	43,474	25,475	10,055.0
		Moz (mix of sources)	1,000	1,000	10,055.0
		Ubersuggest (related)	10,055	4,261	10,055.0
	low carb	Google Keyword Planner	1,378	691	10,055.0
		Ahrefs (all keyword ideas)	319,292	43,488	10,055.0
		Ahrefs (phrase match)	296,019	23,021	10,055.0
		SEMrush (broad match)	188,911	36,523	10,055.0
		SEMrush (phrase match)	180,554	25,475	10,055.0
		Moz (mix of sources)	1,000	1,000	10,055.0
		Ubersuggest (related)	40,634	4,261	10,055.0
	nutrition	Google Keyword Planner	1,025	691	10,055.0
		Ahrefs (all keyword ideas)	1,490,264	43,488	10,055.0
		Ahrefs (phrase match)	1,481,436	23,021	10,055.0
		SEMrush (broad match)	1,317,219	36,523	10,055.0
		SEMrush (phrase match)	1,112,682	25,475	10,055.0
		Moz (mix of sources)	1,000	1,000	10,055.0
		Ubersuggest (related)	223,859	4,261	10,055.0
	diet plans	Google Keyword Planner	1,766	691	10,055.0
		Ahrefs (all keyword ideas)	29,129	43,488	10,055.0
		Ahrefs (phrase match)	6,850	23,021	10,055.0
		SEMrush (broad match)	85,786	36,523	10,055.0
		SEMrush (phrase match)	4,025	25,475	10,055.0
		Moz (mix of sources)	1,000	1,000	10,055.0
		Ubersuggest (related)	1,693	4,261	10,055.0
	weight loss tips	Google Keyword Planner	907	691	10,055.0
		Ahrefs (all keyword ideas)	17,659	43,488	10,055.0
		Ahrefs (phrase match)	2,828	23,021	10,055.0
		SEMrush (broad match)	3,865	36,523	10,055.0
		SEMrush (phrase match)	3,756	25,475	10,055.0
		Moz (mix of sources)	1,000	1,000	10,055.0
		Ubersuggest (related)	1,787	4,261	10,055.0
Online Marketing	marketing	Google Keyword Planner	284	691	34,153.0
		Ahrefs (all keyword ideas)	2,285,404	43,488	34,153.0
		Ahrefs (phrase match)	2,281,425	23,021	34,153.0
		SEMrush (broad match)	5,068,178	36,523	34,153.0
		SEMrush (phrase match)	1,544,835	25,475	34,153.0
		Moz (mix of sources)	1,000	1,000	34,153.0
		Ubersuggest (related)	378,278	4,261	34,153.0
	seo	Google Keyword Planner	428	691	34,153.0
		Ahrefs (all keyword ideas)	362,649	43,488	34,153.0
		Ahrefs (phrase match)	351,981	23,021	34,153.0
		SEMrush (broad match)	234,768	36,523	34,153.0
		SEMrush (phrase match)	229,065	25,475	34,153.0
		Moz (mix of sources)	1,000	1,000	34,153.0
		Ubersuggest (related)	77,664	4,261	34,153.0
	digital marketing	Google Keyword Planner	377	691	34,153.0
		Ahrefs (all keyword ideas)	127,183	43,488	34,153.0
		Ahrefs (phrase match)	106,089	23,021	34,153.0
		SEMrush (broad match)	102,613	36,523	34,153.0
		SEMrush (phrase match)	96,133	25,475	34,153.0
		Moz (mix of sources)	1,000	1,000	34,153.0
		Ubersuggest (related)	30,000	4,261	34,153.0
	affiliate marketing	Google Keyword Planner	1,206	691	34,153.0
		Ahrefs (all keyword ideas)	31,233	43,488	34,153.0
		Ahrefs (phrase match)	25,350	23,021	34,153.0
		SEMrush (broad match)	18,828	36,523	34,153.0
		SEMrush (phrase match)	17,065	25,475	34,153.0
		Moz (mix of sources)	1,000	1,000	34,153.0
		Ubersuggest (related)	6,128	4,261	34,153.0
	marketing plan	Google Keyword Planner	543	691	34,153.0
		Ahrefs (all keyword ideas)	54,207	43,488	34,153.0
		Ahrefs (phrase match)	41,784	23,021	34,153.0
		SEMrush (broad match)	44,816	36,523	34,153.0
		SEMrush (phrase match)	34,153	25,475	34,153.0
		Moz (mix of sources)	1,000	1,000	34,153.0
		Ubersuggest (related)	8,840	4,261	34,153.0
Women Fashion	dresses	Google Keyword Planner	1,545	691	39,866.0
		Ahrefs (all keyword ideas)	1,337,414	43,488	39,866.0
		Ahrefs (phrase match)	1,240,915	23,021	39,866.0
		SEMrush (broad match)	4,447,697	36,523	39,866.0
		SEMrush (phrase match)	767,753	25,475	39,866.0
		Moz (mix of sources)	1,000	1,000	39,866.0
		Ubersuggest (related)	293,656	4,261	39,866.0
	clothes	Google Keyword Planner	2,336	691	39,866.0
		Ahrefs (all keyword ideas)	2,019,714	43,488	39,866.0
		Ahrefs (phrase match)	1,991,817	23,021	39,866.0
		SEMrush (broad match)	2,886,463	36,523	39,866.0
		SEMrush (phrase match)	1,076,138	25,475	39,866.0
		Moz (mix of sources)	1,000	1,000	39,866.0
		Ubersuggest (related)	272,112	4,261	39,866.0
	fashion	Google Keyword Planner	1,290	691	39,866.0
		Ahrefs (all keyword ideas)	1,856,785	43,488	39,866.0
		Ahrefs (phrase match)	1,805,903	23,021	39,866.0
		SEMrush (broad match)	1,386,122	36,523	39,866.0
		SEMrush (phrase match)	1,224,626	25,475	39,866.0
		Moz (mix of sources)	1,000	1,000	39,866.0
		Ubersuggest (related)	310,343	4,261	39,866.0
	cocktail dresses	Google Keyword Planner	1,600	691	39,866.0
		Ahrefs (all keyword ideas)	39,866	43,488	39,866.0
		Ahrefs (phrase match)	14,936	23,021	39,866.0
		SEMrush (broad match)	34,840	36,523	39,866.0
		SEMrush (phrase match)	13,029	25,475	39,866.0
		Moz (mix of sources)	1,000	1,000	39,866.0
		Ubersuggest (related)	4,216	4,261	39,866.0
	plus size dresses	Google Keyword Planner	1,884	691	39,866.0
		Ahrefs (all keyword ideas)	100,985	43,488	39,866.0
		Ahrefs (phrase match)	9,935	23,021	39,866.0
		SEMrush (broad match)	60,499	36,523	39,866.0
		SEMrush (phrase match)	27,851	25,475	39,866.0
		Moz (mix of sources)	1,000	1,000	39,866.0
		Ubersuggest (related)	2,520	4,261	39,866.0
Home Improvement	construction	Google Keyword Planner	369	691	20,392.0
		Ahrefs (all keyword ideas)	2,474,705	43,488	20,392.0
		Ahrefs (phrase match)	2,471,944	23,021	20,392.0
		SEMrush (broad match)	1,964,290	36,523	20,392.0
		SEMrush (phrase match)	1,782,879	25,475	20,392.0
		Moz (mix of sources)	1,000	1,000	20,392.0
		Ubersuggest (related)	292,258	4,261	20,392.0
	renovation	Google Keyword Planner	369	691	20,392.0
		Ahrefs (all keyword ideas)	136,107	43,488	20,392.0
		Ahrefs (phrase match)	133,467	23,021	20,392.0
		SEMrush (broad match)	166,948	36,523	20,392.0
		SEMrush (phrase match)	96,176	25,475	20,392.0
		Moz (mix of sources)	1,000	1,000	20,392.0
		Ubersuggest (related)	21,632	4,261	20,392.0
	bathroom remodel	Google Keyword Planner	677	691	20,392.0
		Ahrefs (all keyword ideas)	25,836	43,488	20,392.0
		Ahrefs (phrase match)	15,058	23,021	20,392.0
		SEMrush (broad match)	20,392	36,523	20,392.0
		SEMrush (phrase match)	12,416	25,475	20,392.0
		Moz (mix of sources)	1,000	1,000	20,392.0
		Ubersuggest (related)	3,241	4,261	20,392.0
	kitchen remodel	Google Keyword Planner	653	691	20,392.0
		Ahrefs (all keyword ideas)	30,068	43,488	20,392.0
		Ahrefs (phrase match)	15,710	23,021	20,392.0
		SEMrush (broad match)	21,215	36,523	20,392.0
		SEMrush (phrase match)	12,296	25,475	20,392.0
		Moz (mix of sources)	1,000	1,000	20,392.0
		Ubersuggest (related)	3,358	4,261	20,392.0
	general contractor	Google Keyword Planner	451	691	20,392.0
		Ahrefs (all keyword ideas)	50,817	43,488	20,392.0
		Ahrefs (phrase match)	47,254	23,021	20,392.0
		SEMrush (broad match)	42,249	36,523	20,392.0
		SEMrush (phrase match)	23,039	25,475	20,392.0
		Moz (mix of sources)	1,000	1,000	20,392.0
		Ubersuggest (related)	4,261	4,261	20,392.0
Gardening	landscaping	Google Keyword Planner	464	691	8,082.0
		Ahrefs (all keyword ideas)	406,570	43,488	8,082.0
		Ahrefs (phrase match)	403,114	23,021	8,082.0
		SEMrush (broad match)	767,343	36,523	8,082.0
		SEMrush (phrase match)	263,772	25,475	8,082.0
		Moz (mix of sources)	1,000	1,000	8,082.0
		Ubersuggest (related)	60,255	4,261	8,082.0
	landscape design	Google Keyword Planner	621	691	8,082.0
		Ahrefs (all keyword ideas)	42,455	43,488	8,082.0
		Ahrefs (phrase match)	28,504	23,021	8,082.0
		SEMrush (broad match)	35,538	36,523	8,082.0
		SEMrush (phrase match)	26,587	25,475	8,082.0
		Moz (mix of sources)	1,000	1,000	8,082.0
		Ubersuggest (related)	6,295	4,261	8,082.0
	garden centre	Google Keyword Planner	182	691	8,082.0
		Ahrefs (all keyword ideas)	37,782	43,488	8,082.0
		Ahrefs (phrase match)	8,001	23,021	8,082.0
		SEMrush (broad match)	53,150	36,523	8,082.0
		SEMrush (phrase match)	47,092	25,475	8,082.0
		Moz (mix of sources)	1,000	1,000	8,082.0
		Ubersuggest (related)	8,082	4,261	8,082.0
	gardening tools	Google Keyword Planner	1,622	691	8,082.0
		Ahrefs (all keyword ideas)	6,037	43,488	8,082.0
		Ahrefs (phrase match)	3,267	23,021	8,082.0
		SEMrush (broad match)	16,215	36,523	8,082.0
		SEMrush (phrase match)	2,016	25,475	8,082.0
		Moz (mix of sources)	1,000	1,000	8,082.0
		Ubersuggest (related)	900	4,261	8,082.0
	plant nursery	Google Keyword Planner	1,342	691	8,082.0
		Ahrefs (all keyword ideas)	33,764	43,488	8,082.0
		Ahrefs (phrase match)	18,008	23,021	8,082.0
		SEMrush (broad match)	26,947	36,523	8,082.0
		SEMrush (phrase match)	18,566	25,475	8,082.0
		Moz (mix of sources)	1,000	1,000	8,082.0
		Ubersuggest (related)	3,720	4,261	8,082.0
Vacations Travel	travel agency	Google Keyword Planner	554	691	16,370.0
		Ahrefs (all keyword ideas)	86,272	43,488	16,370.0
		Ahrefs (phrase match)	64,928	23,021	16,370.0
		SEMrush (broad match)	70,617	36,523	16,370.0
		SEMrush (phrase match)	61,296	25,475	16,370.0
		Moz (mix of sources)	1,000	1,000	16,370.0
		Ubersuggest (related)	19,655	4,261	16,370.0
	all inclusive resorts	Google Keyword Planner	706	691	16,370.0
		Ahrefs (all keyword ideas)	82,001	43,488	16,370.0
		Ahrefs (phrase match)	39,679	23,021	16,370.0
		SEMrush (broad match)	48,465	36,523	16,370.0
		SEMrush (phrase match)	31,456	25,475	16,370.0
		Moz (mix of sources)	1,000	1,000	16,370.0
		Ubersuggest (related)	11,771	4,261	16,370.0
	vacation packages	Google Keyword Planner	709	691	16,370.0
		Ahrefs (all keyword ideas)	106,259	43,488	16,370.0
		Ahrefs (phrase match)	52,750	23,021	16,370.0
		SEMrush (broad match)	45,007	36,523	16,370.0
		SEMrush (phrase match)	36,567	25,475	16,370.0
		Moz (mix of sources)	1,000	1,000	16,370.0
		Ubersuggest (related)	19,248	4,261	16,370.0
	cheap vacations	Google Keyword Planner	1,145	691	16,370.0
		Ahrefs (all keyword ideas)	44,657	43,488	16,370.0
		Ahrefs (phrase match)	4,524	23,021	16,370.0
		SEMrush (broad match)	17,757	36,523	16,370.0
		SEMrush (phrase match)	7,692	25,475	16,370.0
		Moz (mix of sources)	1,000	1,000	16,370.0
		Ubersuggest (related)	1,254	4,261	16,370.0
	travel deals	Google Keyword Planner	1,088	691	16,370.0
		Ahrefs (all keyword ideas)	49,181	43,488	16,370.0
		Ahrefs (phrase match)	15,414	23,021	16,370.0
		SEMrush (broad match)	16,370	36,523	16,370.0
		SEMrush (phrase match)	14,497	25,475	16,370.0
		Moz (mix of sources)	1,000	1,000	16,370.0
		Ubersuggest (related)	5,631	4,261	16,370.0
Legal	legal aid	Google Keyword Planner	326	691	10,094.0
		Ahrefs (all keyword ideas)	73,499	43,488	10,094.0
		Ahrefs (phrase match)	55,473	23,021	10,094.0
		SEMrush (broad match)	45,987	36,523	10,094.0
		SEMrush (phrase match)	44,703	25,475	10,094.0
		Moz (mix of sources)	1,000	1,000	10,094.0
		Ubersuggest (related)	10,114	4,261	10,094.0
	law firm	Google Keyword Planner	342	691	10,094.0
		Ahrefs (all keyword ideas)	294,866	43,488	10,094.0
		Ahrefs (phrase match)	289,472	23,021	10,094.0
		SEMrush (broad match)	276,454	36,523	10,094.0
		SEMrush (phrase match)	224,078	25,475	10,094.0
		Moz (mix of sources)	1,000	1,000	10,094.0
		Ubersuggest (related)	35,696	4,261	10,094.0
	legal separation	Google Keyword Planner	99	691	10,094.0
		Ahrefs (all keyword ideas)	14,505	43,488	10,094.0
		Ahrefs (phrase match)	11,141	23,021	10,094.0
		SEMrush (broad match)	10,094	36,523	10,094.0
		SEMrush (phrase match)	7,075	25,475	10,094.0
		Moz (mix of sources)	1,000	1,000	10,094.0
		Ubersuggest (related)	1,532	4,261	10,094.0
	attorney at law	Google Keyword Planner	788	691	10,094.0
		Ahrefs (all keyword ideas)	38,683	43,488	10,094.0
		Ahrefs (phrase match)	35,483	23,021	10,094.0
		SEMrush (broad match)	23,509	36,523	10,094.0
		SEMrush (phrase match)	19,083	25,475	10,094.0
		Moz (mix of sources)	1,000	1,000	10,094.0
		Ubersuggest (related)	1,311	4,261	10,094.0
	free legal advice	Google Keyword Planner	589	691	10,094.0
		Ahrefs (all keyword ideas)	15,837	43,488	10,094.0
		Ahrefs (phrase match)	6,001	23,021	10,094.0
		SEMrush (broad match)	5,964	36,523	10,094.0
		SEMrush (phrase match)	5,953	25,475	10,094.0
		Moz (mix of sources)	1,000	1,000	10,094.0
		Ubersuggest (related)	1,397	4,261	10,094.0
Office Supplies	office equipment	Google Keyword Planner	680	691	1,434.0
		Ahrefs (all keyword ideas)	17,639	43,488	1,434.0
		Ahrefs (phrase match)	9,452	23,021	1,434.0
		SEMrush (broad match)	8,414	36,523	1,434.0
		SEMrush (phrase match)	7,679	25,475	1,434.0
		Moz (mix of sources)	1,000	1,000	1,434.0
		Ubersuggest (related)	1,527	4,261	1,434.0
	stationery online	Google Keyword Planner	157	691	1,434.0
		Ahrefs (all keyword ideas)	33,696	43,488	1,434.0
		Ahrefs (phrase match)	717	23,021	1,434.0
		SEMrush (broad match)	1,434	36,523	1,434.0
		SEMrush (phrase match)	1,426	25,475	1,434.0
		Moz (mix of sources)	1,000	1,000	1,434.0
		Ubersuggest (related)	432	4,261	1,434.0
	cute office supplies	Google Keyword Planner	55	691	1,434.0
		Ahrefs (all keyword ideas)	2,946	43,488	1,434.0
		Ahrefs (phrase match)	145	23,021	1,434.0
		SEMrush (broad match)	158	36,523	1,434.0
		SEMrush (phrase match)	146	25,475	1,434.0
		Moz (mix of sources)	1,000	1,000	1,434.0
		Ubersuggest (related)	24	4,261	1,434.0
	office furniture	Google Keyword Planner	1,940	691	1,434.0
		Ahrefs (all keyword ideas)	87,178	43,488	1,434.0
		Ahrefs (phrase match)	54,994	23,021	1,434.0
		SEMrush (broad match)	59,647	36,523	1,434.0
		SEMrush (phrase match)	59,224	25,475	1,434.0
		Moz (mix of sources)	1,000	1,000	1,434.0
		Ubersuggest (related)	18,078	4,261	1,434.0
	office supply store	Google Keyword Planner	410	691	1,434.0
		Ahrefs (all keyword ideas)	10,032	43,488	1,434.0
		Ahrefs (phrase match)	3,372	23,021	1,434.0
		SEMrush (broad match)	4,107	36,523	1,434.0
		SEMrush (phrase match)	1,974	25,475	1,434.0
		Moz (mix of sources)	1,000	1,000	1,434.0
		Ubersuggest (related)	250	4,261	1,434.0
Web Hosting	free web hosting	Google Keyword Planner	1,401	691	2,053.0
		Ahrefs (all keyword ideas)	14,315	43,488	2,053.0
		Ahrefs (phrase match)	3,559	23,021	2,053.0
		SEMrush (broad match)	4,315	36,523	2,053.0
		SEMrush (phrase match)	3,972	25,475	2,053.0
		Moz (mix of sources)	1,000	1,000	2,053.0
		Ubersuggest (related)	1,181	4,261	2,053.0
	dedicated server	Google Keyword Planner	665	691	2,053.0
		Ahrefs (all keyword ideas)	60,702	43,488	2,053.0
		Ahrefs (phrase match)	50,602	23,021	2,053.0
		SEMrush (broad match)	28,980	36,523	2,053.0
		SEMrush (phrase match)	25,485	25,475	2,053.0
		Moz (mix of sources)	1,000	1,000	2,053.0
		Ubersuggest (related)	3,763	4,261	2,053.0
	best web hosting	Google Keyword Planner	1,246	691	2,053.0
		Ahrefs (all keyword ideas)	11,931	43,488	2,053.0
		Ahrefs (phrase match)	2,368	23,021	2,053.0
		SEMrush (broad match)	3,358	36,523	2,053.0
		SEMrush (phrase match)	2,902	25,475	2,053.0
		Moz (mix of sources)	1,000	1,000	2,053.0
		Ubersuggest (related)	919	4,261	2,053.0
	domain name registration	Google Keyword Planner	835	691	2,053.0
		Ahrefs (all keyword ideas)	33,962	43,488	2,053.0
		Ahrefs (phrase match)	2,053	23,021	2,053.0
		SEMrush (broad match)	2,103	36,523	2,053.0
		SEMrush (phrase match)	1,943	25,475	2,053.0
		Moz (mix of sources)	1,000	1,000	2,053.0
		Ubersuggest (related)	718	4,261	2,053.0
	cheap domains	Google Keyword Planner	217	691	2,053.0
		Ahrefs (all keyword ideas)	10,757	43,488	2,053.0
		Ahrefs (phrase match)	299	23,021	2,053.0
		SEMrush (broad match)	2,288	36,523	2,053.0
		SEMrush (phrase match)	429	25,475	2,053.0
		Moz (mix of sources)	1,000	1,000	2,053.0
		Ubersuggest (related)	81	4,261	2,053.0
Dating	free dating sites	Google Keyword Planner	757	691	3,951.0
		Ahrefs (all keyword ideas)	49,099	43,488	3,951.0
		Ahrefs (phrase match)	6,838	23,021	3,951.0
		SEMrush (broad match)	24,250	36,523	3,951.0
		SEMrush (phrase match)	15,450	25,475	3,951.0
		Moz (mix of sources)	1,000	1,000	3,951.0
		Ubersuggest (related)	1,946	4,261	3,951.0
	women seeking men	Google Keyword Planner	243	691	3,951.0
		Ahrefs (all keyword ideas)	13,098	43,488	3,951.0
		Ahrefs (phrase match)	7,788	23,021	3,951.0
		SEMrush (broad match)	3,844	36,523	3,951.0
		SEMrush (phrase match)	3,527	25,475	3,951.0
		Moz (mix of sources)	1,000	1,000	3,951.0
		Ubersuggest (related)	842	4,261	3,951.0
	speed dating	Google Keyword Planner	267	691	3,951.0
		Ahrefs (all keyword ideas)	17,715	43,488	3,951.0
		Ahrefs (phrase match)	16,773	23,021	3,951.0
		SEMrush (broad match)	31,566	36,523	3,951.0
		SEMrush (phrase match)	29,304	25,475	3,951.0
		Moz (mix of sources)	1,000	1,000	3,951.0
		Ubersuggest (related)	3,865	4,261	3,951.0
	interracial dating	Google Keyword Planner	194	691	3,951.0
		Ahrefs (all keyword ideas)	8,300	43,488	3,951.0
		Ahrefs (phrase match)	5,120	23,021	3,951.0
		SEMrush (broad match)	3,951	36,523	3,951.0
		SEMrush (phrase match)	3,751	25,475	3,951.0
		Moz (mix of sources)	1,000	1,000	3,951.0
		Ubersuggest (related)	920	4,261	3,951.0
	matrimonial	Google Keyword Planner	533	691	3,951.0
		Ahrefs (all keyword ideas)	24,420	43,488	3,951.0
		Ahrefs (phrase match)	23,021	23,021	3,951.0
		SEMrush (broad match)	47,687	36,523	3,951.0
		SEMrush (phrase match)	15,633	25,475	3,951.0
		Moz (mix of sources)	1,000	1,000	3,951.0
		Ubersuggest (related)	5,913	4,261	3,951.0
Wedding	wedding cakes	Google Keyword Planner	928	691	25,253.5
		Ahrefs (all keyword ideas)	35,360	43,488	25,253.5
		Ahrefs (phrase match)	31,459	23,021	25,253.5
		SEMrush (broad match)	94,548	36,523	25,253.5
		SEMrush (phrase match)	25,475	25,475	25,253.5
		Ubersuggest (related)	7,030	4,261	25,253.5
	wedding invitations	Google Keyword Planner	716	691	25,253.5
		Ahrefs (all keyword ideas)	71,946	43,488	25,253.5
		Ahrefs (phrase match)	55,974	23,021	25,253.5
		SEMrush (broad match)	90,872	36,523	25,253.5
		SEMrush (phrase match)	41,568	25,475	25,253.5
		Moz (mix of sources)	1,000	1,000	25,253.5
		Ubersuggest (related)	14,621	4,261	25,253.5
	wedding planner	Google Keyword Planner	333	691	25,253.5
		Ahrefs (all keyword ideas)	42,023	43,488	25,253.5
		Ahrefs (phrase match)	34,420	23,021	25,253.5
		SEMrush (broad match)	36,523	36,523	25,253.5
		SEMrush (phrase match)	28,913	25,475	25,253.5
		Moz (mix of sources)	1,000	1,000	25,253.5
		Ubersuggest (related)	8,859	4,261	25,253.5
	bridal dresses	Google Keyword Planner	1,390	691	25,253.5
		Ahrefs (all keyword ideas)	39,276	43,488	25,253.5
		Ahrefs (phrase match)	7,634	23,021	25,253.5
		SEMrush (broad match)	25,032	36,523	25,253.5
		SEMrush (phrase match)	12,729	25,475	25,253.5
		Moz (mix of sources)	1,000	1,000	25,253.5
		Ubersuggest (related)	2,490	4,261	25,253.5
	veil	Google Keyword Planner	339	691	25,253.5
		Ahrefs (all keyword ideas)	166,008	43,488	25,253.5
		Ahrefs (phrase match)	163,971	23,021	25,253.5
		SEMrush (broad match)	119,340	36,523	25,253.5
		SEMrush (phrase match)	96,817	25,475	25,253.5
		Moz (mix of sources)	1,000	1,000	25,253.5
		Ubersuggest (related)	14,198	4,261	25,253.5
Automotive	automatic cars	Google Keyword Planner	600	691	5,811.0
		Ahrefs (all keyword ideas)	11,634	43,488	5,811.0
		Ahrefs (phrase match)	3,726	23,021	5,811.0
		SEMrush (broad match)	27,619	36,523	5,811.0
		SEMrush (phrase match)	9,603	25,475	5,811.0
		Moz (mix of sources)	1,000	1,000	5,811.0
		Ubersuggest (related)	1,960	4,261	5,811.0
	motor car	Google Keyword Planner	717	691	5,811.0
		Ahrefs (all keyword ideas)	43,488	43,488	5,811.0
		Ahrefs (phrase match)	7,871	23,021	5,811.0
		SEMrush (broad match)	18,265	36,523	5,811.0
		SEMrush (phrase match)	18,265	25,475	5,811.0
		Moz (mix of sources)	1,000	1,000	5,811.0
		Ubersuggest (related)	1,247	4,261	5,811.0
	sale car	Google Keyword Planner	2,373	691	5,811.0
		Ahrefs (all keyword ideas)	293,908	43,488	5,811.0
		Ahrefs (phrase match)	5,811	23,021	5,811.0
		SEMrush (broad match)	453,534	36,523	5,811.0
		SEMrush (phrase match)	113,664	25,475	5,811.0
		Ubersuggest (related)	526	4,261	5,811.0
	automotive technician	Google Keyword Planner	200	691	5,811.0
		Ahrefs (all keyword ideas)	8,834	43,488	5,811.0
		Ahrefs (phrase match)	4,396	23,021	5,811.0
		SEMrush (broad match)	3,950	36,523	5,811.0
		SEMrush (phrase match)	3,607	25,475	5,811.0
		Ubersuggest (related)	669	4,261	5,811.0
	dealer	Google Keyword Planner	456	691	5,811.0
		Ahrefs (all keyword ideas)	1,052,918	43,488	5,811.0
		Ahrefs (phrase match)	1,037,053	23,021	5,811.0
		SEMrush (broad match)	1,246,425	36,523	5,811.0
		SEMrush (phrase match)	642,171	25,475	5,811.0
		Moz (mix of sources)	1,000	1,000	5,811.0
		Ubersuggest (related)	152,426	4,261	5,811.0
Retail	what is retail	Google Keyword Planner	184	691	2,985.0
		Ahrefs (all keyword ideas)	17,705	43,488	2,985.0
		Ahrefs (phrase match)	2,370	23,021	2,985.0
		SEMrush (broad match)	8,567	36,523	2,985.0
		SEMrush (phrase match)	7,505	25,475	2,985.0
		Moz (mix of sources)	1,000	1,000	2,985.0
		Ubersuggest (related)	315	4,261	2,985.0
	retail marketing	Google Keyword Planner	293	691	2,985.0
		Ahrefs (all keyword ideas)	7,453	43,488	2,985.0
		Ahrefs (phrase match)	2,985	23,021	2,985.0
		SEMrush (broad match)	11,410	36,523	2,985.0
		SEMrush (phrase match)	4,126	25,475	2,985.0
		Moz (mix of sources)	1,000	1,000	2,985.0
		Ubersuggest (related)	817	4,261	2,985.0
	promo code	Google Keyword Planner	2,139	691	2,985.0
		Ahrefs (all keyword ideas)	908,287	43,488	2,985.0
		Ahrefs (phrase match)	855,277	23,021	2,985.0
		SEMrush (broad match)	860,316	36,523	2,985.0
		SEMrush (phrase match)	758,291	25,475	2,985.0
		Moz (mix of sources)	1,000	1,000	2,985.0
		Ubersuggest (related)	184,168	4,261	2,985.0
	voucher codes	Google Keyword Planner	1,546	691	2,985.0
		Ahrefs (all keyword ideas)	11,707	43,488	2,985.0
		Ahrefs (phrase match)	8,562	23,021	2,985.0
		SEMrush (broad match)	62,541	36,523	2,985.0
		SEMrush (phrase match)	15,422	25,475	2,985.0
		Moz (mix of sources)	1,000	1,000	2,985.0
		Ubersuggest (related)	4,487	4,261	2,985.0
	retail trade	Google Keyword Planner	22	691	2,985.0
		Ahrefs (all keyword ideas)	5,067	43,488	2,985.0
		Ahrefs (phrase match)	1,352	23,021	2,985.0
		SEMrush (broad match)	2,652	36,523	2,985.0
		SEMrush (phrase match)	1,662	25,475	2,985.0
		Moz (mix of sources)	1,000	1,000	2,985.0
		Ubersuggest (related)	218	4,261	2,985.0
Solar Energy	solar panels	Google Keyword Planner	916	691	4,710.0
		Ahrefs (all keyword ideas)	168,715	43,488	4,710.0
		Ahrefs (phrase match)	155,362	23,021	4,710.0
		SEMrush (broad match)	185,332	36,523	4,710.0
		SEMrush (phrase match)	86,195	25,475	4,710.0
		Moz (mix of sources)	1,000	1,000	4,710.0
		Ubersuggest (related)	21,147	4,261	4,710.0
	solar battery charger	Google Keyword Planner	797	691	4,710.0
		Ahrefs (all keyword ideas)	12,707	43,488	4,710.0
		Ahrefs (phrase match)	3,223	23,021	4,710.0
		SEMrush (broad match)	5,377	36,523	4,710.0
		SEMrush (phrase match)	5,017	25,475	4,710.0
		Moz (mix of sources)	1,000	1,000	4,710.0
		Ubersuggest (related)	634	4,261	4,710.0
	solar panel price	Google Keyword Planner	1,096	691	4,710.0
		Ahrefs (all keyword ideas)	11,032	43,488	4,710.0
		Ahrefs (phrase match)	1,753	23,021	4,710.0
		SEMrush (broad match)	4,710	36,523	4,710.0
		SEMrush (phrase match)	3,171	25,475	4,710.0
		Moz (mix of sources)	1,000	1,000	4,710.0
		Ubersuggest (related)	1,058	4,261	4,710.0
	free energy	Google Keyword Planner	578	691	4,710.0
		Ahrefs (all keyword ideas)	59,682	43,488	4,710.0
		Ahrefs (phrase match)	46,123	23,021	4,710.0
		SEMrush (broad match)	29,545	36,523	4,710.0
		SEMrush (phrase match)	28,911	25,475	4,710.0
		Moz (mix of sources)	1,000	1,000	4,710.0
		Ubersuggest (related)	2,987	4,261	4,710.0
	sunpower	Google Keyword Planner	388	691	4,710.0
		Ahrefs (all keyword ideas)	16,317	43,488	4,710.0
		Ahrefs (phrase match)	16,317	23,021	4,710.0
		SEMrush (broad match)	9,473	36,523	4,710.0
		SEMrush (phrase match)	9,212	25,475	4,710.0
		Moz (mix of sources)	1,000	1,000	4,710.0
		Ubersuggest (related)	1,631	4,261	4,710.0

To get a sense of the data, we are looking at the distribution of suggestions per SEO tool and keyword category.

Distribution on a Linear Scale

Per SEO Tool:

df_kws %>% 
  ggplot(aes(suggested)) +
    geom_histogram(aes(color = tool_fct, fill = tool_fct), binwidth = 300000) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    coord_cartesian(xlim = c(range_kw[1], range_kw[2])) +
    scale_x_continuous(labels = num_format, expand = c(.08, .08)) +
    scale_color_carto_d(palette = "Prism", guide = F) +
    scale_fill_carto_d(palette = "Prism", guide = F) +
    labs(x = "Keywords Suggestions", y = "Count",
         title = "**Counts of Keyword Suggestions by SEO Tool**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A2_KWS_histo_lin_tool.png"), width = 14, height = 8, dpi = 300) }

Per Keyword Category:

df_kws %>% 
  ggplot(aes(suggested)) +
    geom_histogram(aes(color = category_fct, fill = category_fct), binwidth = 300000) +
    facet_wrap(~ category, scales = "free_x", ncol = 5) +
    coord_cartesian(xlim = c(range_kw[1], range_kw[2])) +
    scale_x_continuous(labels = num_format, expand = c(.1, .1),
                       breaks = c(0, 3000000, 6000000)) +
    scale_color_simpsons(guide = F) +
    scale_fill_simpsons(guide = F) +
    labs(x = "Keywords Suggestions", y = "Count",
         title = "**Counts of Keyword Suggestions by Keyword Category**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A2_KWS_histo_lin_category.png"), width = 14, height = 11, dpi = 300) }

Distribution on a Logarithmic Scale

Since the data is left-skewed (i.e. many more observations with low values compared to higher ones), we are going to use a logarithmic scale for the histograms and most of the following plots.

Per SEO Tool:

df_kws %>% 
  ggplot(aes(suggested)) +
    geom_histogram(aes(color = tool_fct, fill = tool_fct), binwidth = .4) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    coord_cartesian(xlim = c(1, range_kw[2] + 10000000)) +
    scale_x_log10(labels = num_format) +
    scale_color_carto_d(palette = "Prism", guide = F) +
    scale_fill_carto_d(palette = "Prism", guide = F) +
    labs(x = "Keywords suggested", y = "Count",
         title = "**Counts of Keyword Suggestions by SEO Tool**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A2_KWS_histo_log_tool.png"), width = 14, height = 8, dpi = 300) }

Key takeaways:

• Google Keyword Planner reaches very low levels of suggestions with a mean around 1,000.

• Ahrefs and SEMrush reach the highest values of suggestions.

• Moz only returns always a maximum of 1,000 sugestions.
  
  

Per Keyword Category:

df_kws %>% 
  ggplot(aes(suggested)) +
    geom_histogram(aes(color = category_fct, fill = category_fct), binwidth = .4) +
    facet_wrap(~ category, scales = "free_x", ncol = 5) +
    coord_cartesian(xlim = c(1, range_kw[2] + 10000000)) +
    scale_x_log10(labels = num_format) +
    scale_color_simpsons(guide = F) +
    scale_fill_simpsons(guide = F) +
    labs(x = "Keywords suggested", y = "Count",
         title = "**Counts of Keyword Suggestions by Keyword Category**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A2_KWS_histo_log_category.png"), width = 14, height = 11, dpi = 300) }

Key takeaways:

• Distributions of suggested keywords by category range from unimodal (e.g. Web Hosting and Office Supplies) to bimodal (e.g. Vacations Travel, Wedding, and Women Fashion) and more complex patterns.

• Highest range are reached by the keywords Financial Services, Online Marketing, and Women Fashion.
  
  

Raw Data Plots

In the following, we are looking at each search, mapped to SEO tools and keyword categories. The grey, large dot represents the median of each group, here SEO tool which are ranked by this value:

Per Search Eninge:

## jitter per tool
df_kws %>% 
  ggplot(aes(tool_fct, suggested)) + 
    stat_summary(fun.y = median, geom = "point", 
                 size = 8, color = "grey70") +
    geom_jitter(aes(color = category_fct), alpha = .7, width = .3) +
    coord_flip() +
    scale_y_continuous(labels = num_format, expand = c(.02, .02)) +
    scale_color_simpsons(name = "Keyword Category:") +
    guides(color = guide_legend(override.aes = list(size = 5))) +
    labs(x = NULL, y = "Keyword Suggestions",
         title = "**Each Search by SEO Tool**",
         subtitle = "(tools ranked by overall median)") +
    theme_flip + 
    theme(axis.text.y = element_text(size = 13))

if(save == T){ ggsave(here::here("plots", "A2_KWS_jitter_tool.png"), width = 14, height = 10, dpi = 300) }

Key takeaways:

• This plot highlights how skewed the data is - it might be useful to show the large differences between some SEO tools in combination with specific keyword categories(with a focus on SEO tools).

• Even though the dots were jittered, overplotting makes it hard to distinguish each data point in the low-value-area - a log scale helps here.

• Clear ranking when sorted by overall median: Indeed, Ahrefs and SEMrush suggest the highest values, followed by Ubersuggest, Moz, and Google Keyword Planner.
  
  

Per Keyword Category:

## jitter per tool
df_kws %>% 
  ggplot(aes(category_fct, suggested)) + 
    stat_summary(fun.y = median, geom = "point", 
                 size = 6, color = "grey70") +
    #geom_jitter(aes(color = tool_fct), alpha = .7, width = .3) +
    geom_quasirandom(aes(color = tool_fct), 
                     alpha = .7, size = 1.5, 
                     width = .3, bandwidth = .3, varwidth = T) +
    coord_flip() +
    scale_y_continuous(labels = num_format, expand = c(.02, .02)) +
    scale_color_carto_d(palette = "Prism", name = "SEO Tool:") +
    guides(color = guide_legend(override.aes = list(size = 5))) +
    labs(x = NULL, y = "Keyword Suggestions",
         title = "**Each Search by Keyword Category**",
         subtitle = "(categories ranked by overall median)") +
    theme_flip + 
    theme(axis.text.y = element_text(size = 13))

if(save == T){ ggsave(here::here("plots", "A2_KWS_jitter_category.png"), width = 14, height = 10, dpi = 300) }

Key takeaways:

• Indeed, Financial Services, Women Fashion, and Online Marketing gather the most suggestions (ranked 1, 2, and 3 on the y axis), followed by Wedding, Home Improvement, and Vacation Travel.

• Plot highlights how skewed the data is in terms of keyword categories and SEO tools (with a focus on categories).
  
  

Per SEO Tool and Keyword Category on a Logarithmic Scale

Using small multiples, we can visualize the pattern of the raw data per tool and keyword category at the same time:

## facet
df_kws %>% 
  ggplot(aes(category_fct, suggested)) + 
    stat_summary(fun.y = median, geom = "point", 
                 size = 3, color = "grey70") +
    geom_beeswarm(aes(color = category_fct), 
                  alpha = .7, size = 1.5) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    coord_flip() +
    scale_y_log10(labels = num_format, 
                  limits = c(range_kw[1], range_kw[2] + 10000000)) +
    scale_color_simpsons(guide = F) +
    labs(x = NULL, y = "Keyword Suggestions",
         title = "**Each Search per Engine by Keyword Category**",
         subtitle = "(categories ranked by overall median)") +
    theme_flip +
    theme_facet + 
    theme(axis.text.y = element_text(face = "plain"),
          axis.text.x = element_text(size = 9),
          strip.text = element_text(face = "bold"))

if(save == T){ ggsave(here::here("plots", "A2_KWS_bees_facet_tool.png"), width = 14, height = 8, dpi = 300) }

Keywords by Length

Per SEO Tool:

df_kws %>% 
  filter(!str_detect(keyword, "All 5 KW")) %>% 
  mutate(length = nchar(keyword)) %>% 
  ggplot(aes(length, suggested, color = tool_fct, fill = tool_fct)) +
    geom_jitter(width = .1, alpha = .4) +
    geom_smooth(method = "lm", color = "grey7", size = .8, alpha = .2) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    scale_x_continuous(limits = c(1, 25),
                       breaks = c(1, seq(5, 25, by = 5))) +
    scale_y_log10(labels = num_format) +
    scale_color_carto_d(palette = "Prism", guide = F) +
    scale_fill_carto_d(palette = "Prism", guide = F) +
    labs(x = "Number of Keyword Characters", 
         y = "Keyword Suggestions",
         title = "**Trends in Keyword Length per SEO Tool**") +
    theme_facet

if(save == T){ ggsave(here::here("plots", "A2_KWS_length_tool_lm.png"), width = 14, height = 7, dpi = 300) }

df_kws %>% 
  filter(!str_detect(keyword, "All 5 KW")) %>% 
  mutate(length = nchar(keyword)) %>% 
  ggplot(aes(length, suggested, color = tool_fct, fill = tool_fct)) +
    geom_jitter(width = .1, alpha = .4) +
    geom_smooth(color = "grey7", size = .8, alpha = .2) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    scale_x_continuous(limits = c(1, 25),
                       breaks = c(1, seq(5, 25, by = 5))) +
    scale_y_log10(labels = num_format) +
    scale_color_carto_d(palette = "Prism", guide = F) +
    scale_fill_carto_d(palette = "Prism", guide = F) +
    labs(x = "Number of Keyword Characters", 
         y = "Keyword Suggestions",
         title = "**Trends in Keyword Length per SEO Tool**") +
    theme_facet

if(save == T){ ggsave(here::here("plots", "A2_KWS_length_tool_loess.png"), width = 14, height = 7, dpi = 300) }

Key takeaways:

• 
  
  

Per Keyword Category:

df_kws %>% 
  filter(!str_detect(keyword, "All 5 KW")) %>% 
  mutate(length = nchar(keyword)) %>% 
  ggplot(aes(length, suggested, color = category_fct, fill = category_fct)) +
    geom_jitter(width = .1, alpha = .4, size = 2.5) +
    geom_smooth(method = "lm", color = "grey7", size = .8, alpha = .2) +
    facet_wrap(~ category, scales = "free_x", ncol = 5) +
    scale_x_continuous(limits = c(1, 25),
                       breaks = c(1, seq(5, 25, by = 5))) +
    scale_y_log10(labels = num_format) +
    scale_color_simpsons(guide = F) +
    scale_fill_simpsons(guide = F) +
    labs(x = "Number of Keyword Characters", 
         y = "Keyword Suggestions",
         title = "**Trends in Keyword Length per Keyword Category**") +
    theme_facet

if(save == T){ ggsave(here::here("plots", "A2_KWS_length_category_lm.png"), width = 14, height = 10, dpi = 300) }

df_kws %>% 
  filter(!str_detect(keyword, "All 5 KW")) %>% 
  mutate(length = nchar(keyword)) %>% 
  ggplot(aes(length, suggested, color = category_fct, fill = category_fct)) +
    geom_jitter(width = .1, alpha = .4, size = 2.5) +
    geom_smooth(color = "grey7", size = .8, alpha = .2) +
    facet_wrap(~ category, scales = "free_x", ncol = 5) +
    scale_x_continuous(limits = c(1, 25),
                       breaks = c(1, seq(5, 25, by = 5))) +
    scale_y_log10(labels = num_format) +
    scale_color_simpsons(guide = F) +
    scale_fill_simpsons(guide = F) +
    labs(x = "Number of Keyword Characters", 
         y = "Keyword Suggestions",
         title = "**Trends in Keyword Length per Keyword Category**") +
    theme_facet

if(save == T){ ggsave(here::here("plots", "A2_KWS_length_category_loess.png"), width = 14, height = 10, dpi = 300) }

Key takeaways:

• 
  
  

---

3 Keyword Search Volume

Summary Table

## summary table
df_vol %>%
  dplyr::select(
    `SEO Tool` = "tool", 
    volume
  ) %>% 
  group_by(`SEO Tool`) %>% 
  summarize(
    Minimum = min(volume, na.rm = T),
    Average = round(mean(volume, na.rm = T), 1),
    Median = median(volume, na.rm = T),
    Maximum = max(volume, na.rm = T),
    NAs = sum(is.na(volume))
  ) %>% 
  kable(format.args = list(big.mark = ",", 
                           small.mark = ",", 
                           decimal.mark = ".", 
                           scientific = F)) %>%
  kable_styling() %>% 
  column_spec(1, extra_css = "vertical- align:middle;")

SEO Tool	Minimum	Average	Median	Maximum	NAs
Google Keyword Planner	110	15,555.0	3,600	1,500,000	0
Ahrefs	0	9,608.8	2,200	1,780,000	0
SEMrush	20	15,549.2	3,600	1,830,000	0
KeywordTool.io	90	23,483.4	4,400	6,120,000	0
KWFinder	0	14,927.1	3,300	1,390,000	0
LongtailPro	90	15,888.6	3,600	1,500,000	0
SECockpit	70	15,694.7	3,600	1,500,000	0
Sixtrix	25	26,385.1	10,000	250,000	0

Comparison Number of Keywords

df_vol %>% 
  group_by(tool) %>% 
  summarize(n = n()) %>% 
  ggplot(aes(fct_reorder(tool, n), n, 
             fill = tool)) +
    geom_col(color = NA, width = .7) +
    geom_hline(yintercept = seq(0, 10000, by = 2500),
               color = "white", size = .5) +
    coord_flip() +
    scale_y_continuous(labels = num_format, 
                       limits = c(0, 10100),
                       expand = c(.01, .01)) +
    scale_fill_carto_d(palette = "Bold", guide = F) +
    labs(x = NULL, y = "Keywords",
         title = "**Number of Keywords per SEO Tool**") +
    theme_flip +
    theme(panel.grid.major.x = element_blank(),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A3_VOL_bars_tool.png"), width = 14, height = 7, dpi = 300) }

Key takeaways:

• 
  
  

Distribution on a Linear Scale

df_vol %>% 
  ggplot(aes(volume)) +
    geom_histogram(aes(color = tool, fill = tool), binwidth = 200000) +
    facet_wrap(~ tool, scales = "free_x", nrow = 3) +
    coord_cartesian(xlim = c(range_vol[1], range_vol[2])) +
    scale_x_continuous(labels = num_format, expand = c(.1, .1)) +
    scale_color_carto_d(palette = "Bold", guide = F) +
    scale_fill_carto_d(palette = "Bold", guide = F) +
    labs(x = "Search Volume", y = "Count",
         title = "**Distribution of Search Volume by SEO Tool**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A-3_VOL_histo_lin_tool.png"), width = 14, height = 9, dpi = 300) }

Key takeaways:

• 
  
  

Distribution on a Logarithmic Scale

df_vol %>% 
  ggplot(aes(volume)) +
    geom_histogram(aes(color = tool, fill = tool), binwidth = .35) +
    facet_wrap(~ tool, scales = "free_x", nrow = 3) +
    coord_cartesian(xlim = c(1.9, range_vol[2] + 5000000)) +
    scale_x_log10(labels = num_format) +
    scale_color_carto_d(palette = "Bold", guide = F) +
    scale_fill_carto_d(palette = "Bold", guide = F) +
    labs(x = "Search Volume", y = "Count",
         title = "**Distribution of Search Volume by SEO Tool**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A3_VOL_histo_log_tool.png"), width = 14, height = 9, dpi = 300) }

Key takeaways:

• 
  
  

---

4 Keyword Difficulty Scores

Summary Table

## summary table
df_diff %>%
  dplyr::select(
    `SEO Tool` = "tool", 
    diff
  ) %>% 
  group_by(`SEO Tool`) %>% 
  summarize(
    Minimum = round(min(diff, na.rm = T), 2),
    Average = round(mean(diff, na.rm = T), 2),
    Median = round(median(diff, na.rm = T), 2),
    Maximum = round(max(diff, na.rm = T), 2),
    NAs = sum(is.na(diff))
  ) %>%
  kable(format.args = list(big.mark = ",", 
                           small.mark = ",", 
                           decimal.mark = ".", 
                           scientific = F)) %>%
  kable_styling() %>% 
  column_spec(1, extra_css = "vertical- align:middle;")

SEO Tool	Minimum	Average	Median	Maximum	NAs
Ahrefs	0.00	22.54	16.00	99.00	0
SEMrush	0.00	81.17	84.12	99.81	0
KeywordTool.io	0.00	62.62	77.00	100.00	0
KWFinder	6.00	39.14	38.00	86.00	0
LongtailPro	11.00	41.15	41.00	79.00	0
SECockpit	1.29	9.39	7.76	100.00	0
Sixtrix	10.00	50.95	50.00	85.00	0

Comparison Number of Keywords

df_diff %>% 
  group_by(tool) %>% 
  summarize(n = n()) %>% 
  ggplot(aes(fct_reorder(tool, n), n, 
             fill = tool)) +
    geom_col(color = NA, width = .7) +
    geom_hline(yintercept = seq(0, 10000, by = 2500),
               color = "white", size = .5) +
    coord_flip() +
    scale_y_continuous(labels = num_format, 
                       limits = c(0, 10100),
                       expand = c(.01, .01)) +
    scale_fill_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    labs(x = NULL, y = "Keywords",
         title = "**Number of Keywords per SEO Tool**") +
    theme_flip +
    theme(panel.grid.major.x = element_blank(),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A4_DIF_bars_tool.png"), width = 14, height = 6, dpi = 300) }

Key takeaways:

• 
  
  

Distribution on a Linear Scale

df_diff %>% 
  ggplot(aes(diff)) +
    geom_histogram(aes(color = tool, fill = tool), binwidth = 5) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    coord_cartesian(xlim = c(-2, range_diff[2]), expand = c(.1, .1)) +
    scale_x_continuous(labels = num_format) +
    scale_color_carto_d(palette = "Bold", guide = F, 
                        limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    labs(x = "Difficulty Score", y = "Count",
         title = "**Distribution of Difficulty Scores by SEO Tool**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A4_DIF_histo_lin_tool.png"), width = 14, height = 7, dpi = 300) }

Key takeaways:

• 
  
  

Distribution on a Logarithmic Scale

df_diff %>% 
  ggplot(aes(diff)) +
    geom_histogram(aes(color = tool, fill = tool), binwidth = .1) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    coord_cartesian(xlim = c(.9, range_diff[2])) +
    scale_x_log10(labels = num_format) +
    scale_color_carto_d(palette = "Bold", guide = F, 
                        limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    labs(x = "Difficulty Score", y = "Count",
         title = "**Distribution of Difficulty Scores by SEO Tool**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A4_DIF_histo_log_tool.png"), width = 14, height = 7, dpi = 300) }

Key takeaways:

• 
  
  

---

5 Cost per Click (CPC)

Summary Table

df_cpc %>%
  dplyr::select(
    `SEO Tool` = "tool", 
    cpc
  ) %>% 
  group_by(`SEO Tool`) %>% 
  summarize(
    Minimum = round(min(cpc, na.rm = T), 2),
    Average = round(mean(cpc, na.rm = T), 2),
    Median = round(median(cpc, na.rm = T), 2),
    Maximum = round(max(cpc, na.rm = T), 2),
    NAs = sum(is.na(cpc))
  ) %>% 
  kable(format.args = list(big.mark = ",", 
                           small.mark = ",", 
                           decimal.mark = ".", 
                           scientific = F)) %>%
  kable_styling() %>% 
  column_spec(1, extra_css = "vertical- align:middle;")

SEO Tool	Minimum	Average	Median	Maximum	NAs
Google Keyword Planner	0.02	5.03	2.12	749.06	0
Ahrefs	0.01	3.63	1.60	355.00	0
SEMrush	0.01	3.12	1.38	209.10	0
KeywordTool.io	0.02	4.00	1.60	361.05	0
KWFinder	0.02	4.46	1.66	361.05	0
LongtailPro	0.01	4.33	1.61	361.05	0
SECockpit	0.01	5.25	2.20	502.10	0
Sixtrix	0.11	5.07	1.66	195.47	0

Comparison Number of Keywords

df_cpc %>% 
  group_by(tool) %>% 
  summarize(n = n()) %>% 
  ggplot(aes(fct_reorder(tool, n), n, 
             fill = tool)) +
    geom_col(color = NA, width = .7) +
    geom_hline(yintercept = seq(0, 10000, by = 2500),
               color = "white", size = .5) +
    coord_flip() +
    scale_y_continuous(labels = num_format, 
                       limits = c(0, 10100),
                       expand = c(.01, .01)) +
    scale_fill_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    labs(x = NULL, y = "Keywords",
         title = "**Number of Keywords per SEO Tool**") +
    theme_flip +
    theme(panel.grid.major.x = element_blank(),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A5_CPC_bars_tool.png"), width = 14, height = 7, dpi = 300) }

Key takeaways:

• 
  
  

Distribution on a Linear Scale

df_cpc %>% 
  ggplot(aes(cpc)) +
    geom_histogram(aes(color = tool, fill = tool), binwidth = 30) +
    facet_wrap(~ tool, scales = "free_x", nrow = 2) +
    coord_cartesian(xlim = c(-20, range_cpc[2], expand = c(.1, .1))) +
    scale_x_continuous(labels = num_format) +
    scale_color_carto_d(palette = "Bold", guide = F, 
                        limits = levels(df_tools$tool)) +
    scale_fill_carto_d(palette = "Bold", guide = F, 
                       limits = levels(df_tools$tool)) +
    labs(x = "CPC", y = "Count",
         title = "**Distribution of Cost per Click (CPC) by SEO Tool**") +
    theme_facet +
    theme(panel.grid.major.y = element_line(size = .25),
          axis.text.x = element_text(size = 8))

if(save == T){ ggsave(here::here("plots", "A5_CPC_histo_lin_tool.png"), width = 14, height = 7, dpi = 300) }

Key takeaways:

• 
  
  

---

Session Info

## R version 3.6.2 (2019-12-12)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 17763)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=German_Germany.1252  LC_CTYPE=German_Germany.1252   
## [3] LC_MONETARY=German_Germany.1252 LC_NUMERIC=C                   
## [5] LC_TIME=German_Germany.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] patchwork_1.0.0.9000 ggsci_2.9            rcartocolor_2.0.0   
##  [4] kableExtra_1.1.0     ggtext_0.1.0         ggbeeswarm_0.6.0    
##  [7] showtext_0.7         showtextdb_2.0       sysfonts_0.8        
## [10] janitor_1.2.0        rmdformats_0.3.6     inspectdf_0.0.7     
## [13] DataExplorer_0.8.1   knitr_1.26           fs_1.3.1            
## [16] imager_0.41.2        magrittr_1.5         magick_2.2          
## [19] glue_1.3.1           treemapify_2.5.3     reshape2_1.4.3      
## [22] prettydoc_0.3.1      mblm_0.12.1          sm_2.2-5.6          
## [25] DT_0.10              sentimentr_2.7.1     tidytext_0.2.2      
## [28] here_0.1             gganimate_1.0.4      scales_1.1.0        
## [31] plotly_4.9.1         extrafont_0.17       hrbrthemes_0.7.1    
## [34] ggthemes_4.2.0       readxl_1.3.1         forcats_0.4.0       
## [37] stringr_1.4.0        dplyr_0.8.3          purrr_0.3.3         
## [40] readr_1.3.1          tidyr_1.0.0          tibble_2.1.3        
## [43] ggplot2_3.2.1        tidyverse_1.3.0     
## 
## loaded via a namespace (and not attached):
##  [1] backports_1.1.5   systemfonts_0.1.1 selectr_0.4-2     plyr_1.8.5       
##  [5] igraph_1.2.4.2    lazyeval_0.2.2    SnowballC_0.6.0   digest_0.6.23    
##  [9] htmltools_0.4.0   tiff_0.1-5        fansi_0.4.0       ggfittext_0.8.1  
## [13] modelr_0.1.5      extrafontdb_1.0   prettyunits_1.0.2 jpeg_0.1-8.1     
## [17] colorspace_1.4-1  rvest_0.3.5       haven_2.2.0       xfun_0.11        
## [21] tcltk_3.6.2       crayon_1.3.4      jsonlite_1.6      zeallot_0.1.0    
## [25] gtable_0.3.0      webshot_0.5.2     scico_1.1.0       Rttf2pt1_1.3.7   
## [29] DBI_1.0.0         qdapRegex_0.7.2   Rcpp_1.0.3        viridisLite_0.3.0
## [33] progress_1.2.2    gridtext_0.1.0    textclean_0.9.3   htmlwidgets_1.5.1
## [37] httr_1.4.1        ellipsis_0.3.0    pkgconfig_2.0.3   farver_2.0.1     
## [41] dbplyr_1.4.2      tidyselect_0.2.5  labeling_0.3      rlang_0.4.2      
## [45] munsell_0.5.0     cellranger_1.1.0  tools_3.6.2       cli_2.0.0        
## [49] generics_0.0.2    gifski_0.8.6      broom_0.5.2       evaluate_0.14    
## [53] yaml_2.2.0        readbitmap_0.1.5  nlme_3.1-142      xml2_1.2.2       
## [57] tokenizers_0.2.1  compiler_3.6.2    rstudioapi_0.10   beeswarm_0.2.3   
## [61] curl_4.3          png_0.1-7         reprex_0.3.0      syuzhet_1.0.4    
## [65] tweenr_1.0.1      stringi_1.4.3     highr_0.8         gdtools_0.2.1    
## [69] lattice_0.20-38   Matrix_1.2-18     markdown_1.1      vctrs_0.2.0      
## [73] pillar_1.4.2      lifecycle_0.1.0   networkD3_0.4     data.table_1.12.8
## [77] R6_2.4.1          bookdown_0.16     gridExtra_2.3     bmp_0.3          
## [81] vipor_0.4.5       janeaustenr_0.1.5 lexicon_1.2.1     assertthat_0.2.1 
## [85] rprojroot_1.3-2   withr_2.1.2       parallel_3.6.2    hms_0.5.2        
## [89] grid_3.6.2        rmarkdown_2.0     snakecase_0.11.0  lubridate_1.7.4